nixbot

1these 13 derivations will be built:2  /nix/store/gpks4yx37b6ya0k0njw1b3sfwkhn1wck-python3.13-nixbot.drv3  /nix/store/0lrd76q4a17aiiwp3sc00cgrfaj11psf-unit-nixbot.service.drv4  /nix/store/jmdzj5vmn651r4zipnjyzwq7d4ks1fq1-system-units.drv5  /nix/store/z7jzbhj24bjnpa1s5yaknbc752m9d7gy-etc.drv6  /nix/store/787aa31zl6wxhh460c26abhy4mkjhgim-activate.drv7  /nix/store/fl56cpgdgk86lk5qszl607f5m6xbx4pm-nixos-system-gitlab-test.drv8  /nix/store/bm7nvd8pd5rwr76kkd0ar38j30cfi5p7-closure-info.drv9  /nix/store/30jz2pxf6daghin1vbifpr8v03c3r6sr-closure-info.drv10  /nix/store/nbwbcj74psal25jyj38g8qnb4a35c5rm-run-nixos-vm.drv11  /nix/store/l0w22k8h94v9qkhbfs1vjg4cifmgm12p-nixos-vm.drv12  /nix/store/3zrlzb9gspxywaf4biy0mnyf3l4vadrc-driverConfiguration.json.drv13  /nix/store/fiaxd1l66dfp7gn1bgyx10c2j7am1k1r-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/3shci936yvl7libv0vlwfp6rfmdfw3i3-vm-test-run-nixbot-gitlab.drv15building '/nix/store/gpks4yx37b6ya0k0njw1b3sfwkhn1wck-python3.13-nixbot.drv' on 'ssh-ng://nix@jamie'16building '/nix/store/gpks4yx37b6ya0k0njw1b3sfwkhn1wck-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/p72waiwbb9ncv7qg16isi35ybl3jgac7-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot/lib /nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/g2a1p1bdcshnpwiagcd7n73d2475zfvv-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/g2a1p1bdcshnpwiagcd7n73d2475zfvv-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/g2a1p1bdcshnpwiagcd7n73d2475zfvv-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/b11c82p9j3xm1g6q3zl1ywjhfi26wbg0-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/0lrd76q4a17aiiwp3sc00cgrfaj11psf-unit-nixbot.service.drv'75unit-nixbot.service> tribuchet: building on jamie76building '/nix/store/jmdzj5vmn651r4zipnjyzwq7d4ks1fq1-system-units.drv'77system-units> tribuchet: building on jamie78building '/nix/store/z7jzbhj24bjnpa1s5yaknbc752m9d7gy-etc.drv'79etc> tribuchet: building on jamie80building '/nix/store/787aa31zl6wxhh460c26abhy4mkjhgim-activate.drv'81activate> tribuchet: building on jamie82building '/nix/store/fl56cpgdgk86lk5qszl607f5m6xbx4pm-nixos-system-gitlab-test.drv'83nixos-system-gitlab-test> tribuchet: building on jamie84nixos-system-gitlab-test> structuredAttrs is enabled85building '/nix/store/bm7nvd8pd5rwr76kkd0ar38j30cfi5p7-closure-info.drv'86closure-info> tribuchet: building on jamie87closure-info> structuredAttrs is enabled88building '/nix/store/30jz2pxf6daghin1vbifpr8v03c3r6sr-closure-info.drv'89closure-info> tribuchet: building on jamie90closure-info> structuredAttrs is enabled91building '/nix/store/nbwbcj74psal25jyj38g8qnb4a35c5rm-run-nixos-vm.drv'92run-nixos-vm> tribuchet: building on jamie93building '/nix/store/l0w22k8h94v9qkhbfs1vjg4cifmgm12p-nixos-vm.drv'94nixos-vm> tribuchet: building on jamie95building '/nix/store/3zrlzb9gspxywaf4biy0mnyf3l4vadrc-driverConfiguration.json.drv'96driverConfiguration.json> tribuchet: building on jamie97driverConfiguration.json> structuredAttrs is enabled98building '/nix/store/fiaxd1l66dfp7gn1bgyx10c2j7am1k1r-nixos-test-driver-nixbot-gitlab.drv'99nixos-test-driver-nixbot-gitlab> tribuchet: building on jamie100nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)101nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck102nixos-test-driver-nixbot-gitlab> All checks passed!103nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint105nixos-test-driver-nixbot-gitlab> All checks passed!106building '/nix/store/3shci936yvl7libv0vlwfp6rfmdfw3i3-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie'107building '/nix/store/3shci936yvl7libv0vlwfp6rfmdfw3i3-vm-test-run-nixbot-gitlab.drv'108vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state109vm-test-run-nixbot-gitlab> start all VLans110vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)111vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds112vm-test-run-nixbot-gitlab> run the VM test script113vm-test-run-nixbot-gitlab> additionally exposed symbols:114vm-test-run-nixbot-gitlab>     gitlab,115vm-test-run-nixbot-gitlab>     vlan1,116vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh117vm-test-run-nixbot-gitlab> start all VMs118vm-test-run-nixbot-gitlab> gitlab: starting vm119vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...120vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)121vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.Njc6Qe4P71', fmt=raw size=1073741824122vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)123vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done124vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes125vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 393e42d7-bbea-4364-93d9-c419e048f38d126vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:127vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376128vm-test-run-nixbot-gitlab> gitlab # 129vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done130vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done131vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done132vm-test-run-nixbot-gitlab> (finished: start all VMs, in 33.80 seconds)133vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done134vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service135vm-test-run-nixbot-gitlab> gitlab # 136vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.137vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...138vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.139vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting140vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)141vm-test-run-nixbot-gitlab> gitlab # 142vm-test-run-nixbot-gitlab> gitlab # 143vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00144vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...145vm-test-run-nixbot-gitlab> gitlab # 146vm-test-run-nixbot-gitlab> gitlab # 147vm-test-run-nixbot-gitlab> gitlab # 148vm-test-run-nixbot-gitlab> gitlab # 149vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00150vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...151vm-test-run-nixbot-gitlab> gitlab # 152vm-test-run-nixbot-gitlab> gitlab # 153vm-test-run-nixbot-gitlab> gitlab # Booting from ROM...154vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok[    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun  1 15:51:08 UTC 2026155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xgnv8xfbgl3ni7xkxqwfk9i51v9ywqly-nixos-system-gitlab-test/init regInfo=/nix/store/3nw9vmd2583ivydq8z42ixkx98gmz93d-closure-info/registration console=ttyS0,115200n8 console=tty0156vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-provided physical RAM map:157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable165vm-test-run-nixbot-gitlab> gitlab # [    0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved166vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NX (Execute Disable) protection: active167vm-test-run-nixbot-gitlab> gitlab # [    0.000000] APIC: Static calls initialized168vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SMBIOS 2.8 present.169vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014170vm-test-run-nixbot-gitlab> gitlab # [    0.000000] DMI: Memory slots populated: 1/1171vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Hypervisor detected: KVM172vm-test-run-nixbot-gitlab> gitlab # [    0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000173vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00174vm-test-run-nixbot-gitlab> gitlab # [    0.000000] kvm-clock: using sched offset of 500051840 cycles175vm-test-run-nixbot-gitlab> gitlab # [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns176vm-test-run-nixbot-gitlab> gitlab # [    0.000005] tsc: Detected 2400.010 MHz processor177vm-test-run-nixbot-gitlab> gitlab # [    0.000815] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000178vm-test-run-nixbot-gitlab> gitlab # [    0.000852] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs179vm-test-run-nixbot-gitlab> gitlab # [    0.000855] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT180vm-test-run-nixbot-gitlab> gitlab # [    0.000906] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000181vm-test-run-nixbot-gitlab> gitlab # [    0.002769] found SMP MP-table at [mem 0x000f5470-0x000f547f]182vm-test-run-nixbot-gitlab> gitlab # [    0.002780] Using GB pages for direct mapping183vm-test-run-nixbot-gitlab> gitlab # [    0.002950] RAMDISK: [mem 0xbe4e0000-0xbffcffff]184vm-test-run-nixbot-gitlab> gitlab # [    0.002956] ACPI: Early table checksum verification disabled185vm-test-run-nixbot-gitlab> gitlab # [    0.002960] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )186vm-test-run-nixbot-gitlab> gitlab # [    0.002963] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS  BXPC     00000001 BXPC 00000001)187vm-test-run-nixbot-gitlab> gitlab # [    0.002967] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS  BXPC     00000001 BXPC 00000001)188vm-test-run-nixbot-gitlab> gitlab # [    0.002974] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS  BXPC     00000001 BXPC 00000001)189vm-test-run-nixbot-gitlab> gitlab # [    0.002976] ACPI: FACS 0x00000000BFFE0000 000040190vm-test-run-nixbot-gitlab> gitlab # [    0.002978] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS  BXPC     00000001 BXPC 00000001)191vm-test-run-nixbot-gitlab> gitlab # [    0.002980] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS  BXPC     00000001 BXPC 00000001)192vm-test-run-nixbot-gitlab> gitlab # [    0.002981] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS  BXPC     00000001 BXPC 00000001)193vm-test-run-nixbot-gitlab> gitlab # [    0.002983] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]194vm-test-run-nixbot-gitlab> gitlab # [    0.002984] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]195vm-test-run-nixbot-gitlab> gitlab # [    0.002984] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]196vm-test-run-nixbot-gitlab> gitlab # [    0.002985] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]197vm-test-run-nixbot-gitlab> gitlab # [    0.002985] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]198vm-test-run-nixbot-gitlab> gitlab # [    0.002986] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]199vm-test-run-nixbot-gitlab> gitlab # [    0.003468] No NUMA configuration found200vm-test-run-nixbot-gitlab> gitlab # [    0.003470] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]201vm-test-run-nixbot-gitlab> gitlab # [    0.003473] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]202vm-test-run-nixbot-gitlab> gitlab # [    0.005898] Zone ranges:203vm-test-run-nixbot-gitlab> gitlab # [    0.005899]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]204vm-test-run-nixbot-gitlab> gitlab # [    0.005901]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]205vm-test-run-nixbot-gitlab> gitlab # [    0.005902]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]206vm-test-run-nixbot-gitlab> gitlab # [    0.005903]   Device   empty207vm-test-run-nixbot-gitlab> gitlab # [    0.005903] Movable zone start for each node208vm-test-run-nixbot-gitlab> gitlab # [    0.005904] Early memory node ranges209vm-test-run-nixbot-gitlab> gitlab # [    0.005905]   node   0: [mem 0x0000000000001000-0x000000000009efff]210vm-test-run-nixbot-gitlab> gitlab # [    0.005906]   node   0: [mem 0x0000000000100000-0x00000000bffd7fff]211vm-test-run-nixbot-gitlab> gitlab # [    0.005907]   node   0: [mem 0x0000000100000000-0x00000001bfffffff]212vm-test-run-nixbot-gitlab> gitlab # [    0.005908] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]213vm-test-run-nixbot-gitlab> gitlab # [    0.005927] On node 0, zone DMA: 1 pages in unavailable ranges214vm-test-run-nixbot-gitlab> gitlab # [    0.006212] On node 0, zone DMA: 97 pages in unavailable ranges215vm-test-run-nixbot-gitlab> gitlab # [    0.119716] On node 0, zone Normal: 40 pages in unavailable ranges216vm-test-run-nixbot-gitlab> gitlab # [    0.120752] ACPI: PM-Timer IO Port: 0x608217vm-test-run-nixbot-gitlab> gitlab # [    0.120769] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])218vm-test-run-nixbot-gitlab> gitlab # [    0.120807] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23219vm-test-run-nixbot-gitlab> gitlab # [    0.120809] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)220vm-test-run-nixbot-gitlab> gitlab # [    0.120811] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)221vm-test-run-nixbot-gitlab> gitlab # [    0.120812] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)222vm-test-run-nixbot-gitlab> gitlab # [    0.120813] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)223vm-test-run-nixbot-gitlab> gitlab # [    0.120814] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)224vm-test-run-nixbot-gitlab> gitlab # [    0.120817] ACPI: Using ACPI (MADT) for SMP configuration information225vm-test-run-nixbot-gitlab> gitlab # [    0.120818] ACPI: HPET id: 0x8086a201 base: 0xfed00000226vm-test-run-nixbot-gitlab> gitlab # [    0.120825] TSC deadline timer available227vm-test-run-nixbot-gitlab> gitlab # [    0.120829] CPU topo: Max. logical packages:   1228vm-test-run-nixbot-gitlab> gitlab # [    0.120830] CPU topo: Max. logical dies:       1229vm-test-run-nixbot-gitlab> gitlab # [    0.120830] CPU topo: Max. dies per package:   1230vm-test-run-nixbot-gitlab> gitlab # [    0.120834] CPU topo: Max. threads per core:   1231vm-test-run-nixbot-gitlab> gitlab # [    0.120834] CPU topo: Num. cores per package:     4232vm-test-run-nixbot-gitlab> gitlab # [    0.120835] CPU topo: Num. threads per package:   4233vm-test-run-nixbot-gitlab> gitlab # [    0.120835] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs234vm-test-run-nixbot-gitlab> gitlab # [    0.120862] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()235vm-test-run-nixbot-gitlab> gitlab # [    0.120881] kvm-guest: KVM setup pv remote TLB flush236vm-test-run-nixbot-gitlab> gitlab # [    0.120885] kvm-guest: setup PV sched yield237vm-test-run-nixbot-gitlab> gitlab # [    0.120908] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]238vm-test-run-nixbot-gitlab> gitlab # [    0.120910] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]239vm-test-run-nixbot-gitlab> gitlab # [    0.120911] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]240vm-test-run-nixbot-gitlab> gitlab # [    0.120912] [mem 0xc0000000-0xfeffbfff] available for PCI devices241vm-test-run-nixbot-gitlab> gitlab # [    0.120914] Booting paravirtualized kernel on KVM242vm-test-run-nixbot-gitlab> gitlab # [    0.120918] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns243vm-test-run-nixbot-gitlab> gitlab # [    0.125410] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1244vm-test-run-nixbot-gitlab> gitlab # [    0.127850] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288245vm-test-run-nixbot-gitlab> gitlab # [    0.127900] kvm-guest: PV spinlocks enabled246vm-test-run-nixbot-gitlab> gitlab # [    0.127902] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)247vm-test-run-nixbot-gitlab> gitlab # [    0.127905] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xgnv8xfbgl3ni7xkxqwfk9i51v9ywqly-nixos-system-gitlab-test/init regInfo=/nix/store/3nw9vmd2583ivydq8z42ixkx98gmz93d-closure-info/registration console=ttyS0,115200n8 console=tty0248vm-test-run-nixbot-gitlab> gitlab # [    0.128003] Unknown kernel command line parameters "regInfo=/nix/store/3nw9vmd2583ivydq8z42ixkx98gmz93d-closure-info/registration", will be passed to user space.249vm-test-run-nixbot-gitlab> gitlab # [    0.128016] random: crng init done250vm-test-run-nixbot-gitlab> gitlab # [    0.128016] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes251vm-test-run-nixbot-gitlab> gitlab # [    0.137008] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)252vm-test-run-nixbot-gitlab> gitlab # [    0.141418] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)253vm-test-run-nixbot-gitlab> gitlab # [    0.141497] software IO TLB: area num 4.254vm-test-run-nixbot-gitlab> gitlab # [    0.216803] Fallback order for Node 0: 0255vm-test-run-nixbot-gitlab> gitlab # [    0.216812] Built 1 zonelists, mobility grouping on.  Total pages: 1572726256vm-test-run-nixbot-gitlab> gitlab # [    0.216814] Policy zone: Normal257vm-test-run-nixbot-gitlab> gitlab # [    0.219603] mem auto-init: stack:all(zero), heap alloc:on, heap free:off258vm-test-run-nixbot-gitlab> gitlab # [    0.229816] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1259vm-test-run-nixbot-gitlab> gitlab # [    0.243437] allocated 12582912 bytes of page_ext260vm-test-run-nixbot-gitlab> gitlab # [    0.253615] ftrace: allocating 48565 entries in 192 pages261vm-test-run-nixbot-gitlab> gitlab # [    0.253616] ftrace: allocated 192 pages with 2 groups262vm-test-run-nixbot-gitlab> gitlab # [    0.254469] Dynamic Preempt: lazy263vm-test-run-nixbot-gitlab> gitlab # [    0.254656] rcu: Preemptible hierarchical RCU implementation.264vm-test-run-nixbot-gitlab> gitlab # [    0.254656] rcu: 	RCU event tracing is enabled.265vm-test-run-nixbot-gitlab> gitlab # [    0.254657] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.266vm-test-run-nixbot-gitlab> gitlab # [    0.254659] 	Trampoline variant of Tasks RCU enabled.267vm-test-run-nixbot-gitlab> gitlab # [    0.254659] 	Rude variant of Tasks RCU enabled.268vm-test-run-nixbot-gitlab> gitlab # [    0.254659] 	Tracing variant of Tasks RCU enabled.269vm-test-run-nixbot-gitlab> gitlab # [    0.254660] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.270vm-test-run-nixbot-gitlab> gitlab # [    0.254661] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4271vm-test-run-nixbot-gitlab> gitlab # [    0.254681] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.272vm-test-run-nixbot-gitlab> gitlab # [    0.254682] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.273vm-test-run-nixbot-gitlab> gitlab # [    0.254683] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.274vm-test-run-nixbot-gitlab> gitlab # [    0.258992] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16275vm-test-run-nixbot-gitlab> gitlab # [    0.259324] rcu: srcu_init: Setting srcu_struct sizes based on contention.276vm-test-run-nixbot-gitlab> gitlab # [    0.259444] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)277vm-test-run-nixbot-gitlab> gitlab # [    0.266768] Console: colour VGA+ 80x25278vm-test-run-nixbot-gitlab> gitlab # [    0.266772] printk: legacy console [tty0] enabled279vm-test-run-nixbot-gitlab> gitlab # [    0.310749] printk: legacy console [ttyS0] enabled280vm-test-run-nixbot-gitlab> gitlab # [    0.512224] ACPI: Core revision 20250807281vm-test-run-nixbot-gitlab> gitlab # [    0.513819] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns282vm-test-run-nixbot-gitlab> gitlab # [    0.516682] APIC: Switch to symmetric I/O mode setup283vm-test-run-nixbot-gitlab> gitlab # [    0.518461] x2apic enabled284vm-test-run-nixbot-gitlab> gitlab # [    0.519734] APIC: Switched APIC routing to: physical x2apic285vm-test-run-nixbot-gitlab> gitlab # [    0.521442] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()286vm-test-run-nixbot-gitlab> gitlab # [    0.523661] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()287vm-test-run-nixbot-gitlab> gitlab # [    0.526433] kvm-guest: setup PV IPIs288vm-test-run-nixbot-gitlab> gitlab # [    0.529977] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1289vm-test-run-nixbot-gitlab> gitlab # [    0.531864] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns290vm-test-run-nixbot-gitlab> gitlab # [    0.534982] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)291vm-test-run-nixbot-gitlab> gitlab # [    0.536097] x86/cpu: User Mode Instruction Prevention (UMIP) activated292vm-test-run-nixbot-gitlab> gitlab # [    0.539067] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127293vm-test-run-nixbot-gitlab> gitlab # [    0.539981] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0294vm-test-run-nixbot-gitlab> gitlab # [    0.541985] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto295vm-test-run-nixbot-gitlab> gitlab # [    0.543980] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl296vm-test-run-nixbot-gitlab> gitlab # [    0.544981] Transient Scheduler Attacks: Mitigation: Clear CPU buffers297vm-test-run-nixbot-gitlab> gitlab # [    0.546980] Spectre V2 : Mitigation: Enhanced / Automatic IBRS298vm-test-run-nixbot-gitlab> gitlab # [    0.548980] Speculative Return Stack Overflow: Mitigation: Safe RET299vm-test-run-nixbot-gitlab> gitlab # [    0.549980] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization300vm-test-run-nixbot-gitlab> gitlab # [    0.550986] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier301vm-test-run-nixbot-gitlab> gitlab # [    0.551980] active return thunk: srso_alias_return_thunk302vm-test-run-nixbot-gitlab> gitlab # [    0.553009] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'303vm-test-run-nixbot-gitlab> gitlab # [    0.553980] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'304vm-test-run-nixbot-gitlab> gitlab # [    0.554980] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'305vm-test-run-nixbot-gitlab> gitlab # [    0.555980] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'306vm-test-run-nixbot-gitlab> gitlab # [    0.556980] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'307vm-test-run-nixbot-gitlab> gitlab # [    0.557980] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'308vm-test-run-nixbot-gitlab> gitlab # [    0.558980] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'309vm-test-run-nixbot-gitlab> gitlab # [    0.559980] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers'310vm-test-run-nixbot-gitlab> gitlab # [    0.560980] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)'311vm-test-run-nixbot-gitlab> gitlab # [    0.562980] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256312vm-test-run-nixbot-gitlab> gitlab # [    0.563980] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64313vm-test-run-nixbot-gitlab> gitlab # [    0.564980] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512314vm-test-run-nixbot-gitlab> gitlab # [    0.565980] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024315vm-test-run-nixbot-gitlab> gitlab # [    0.566980] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:    8316vm-test-run-nixbot-gitlab> gitlab # [    0.568980] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]:   16317vm-test-run-nixbot-gitlab> gitlab # [    0.570980] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]:   24318vm-test-run-nixbot-gitlab> gitlab # [    0.572980] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format.319vm-test-run-nixbot-gitlab> gitlab # [    0.603812] Freeing SMP alternatives memory: 44K320vm-test-run-nixbot-gitlab> gitlab # [    0.604982] pid_max: default: 32768 minimum: 301321vm-test-run-nixbot-gitlab> gitlab # [    0.606104] LSM: initializing lsm=capability,landlock,yama,bpf,ima322vm-test-run-nixbot-gitlab> gitlab # [    0.608088] landlock: Up and running.323vm-test-run-nixbot-gitlab> gitlab # [    0.609980] Yama: becoming mindful.324vm-test-run-nixbot-gitlab> gitlab # [    0.611217] LSM support for eBPF active325vm-test-run-nixbot-gitlab> gitlab # [    0.612987] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)326vm-test-run-nixbot-gitlab> gitlab # [    0.614144] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)327vm-test-run-nixbot-gitlab> gitlab # [    0.617277] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)328vm-test-run-nixbot-gitlab> gitlab # [    0.618652] Performance Events: Fam17h+ core perfctr, AMD PMU driver.329vm-test-run-nixbot-gitlab> gitlab # [    0.619991] ... version:                   2330vm-test-run-nixbot-gitlab> gitlab # [    0.620982] ... bit width:                 48331vm-test-run-nixbot-gitlab> gitlab # [    0.621982] ... generic counters:          6332vm-test-run-nixbot-gitlab> gitlab # [    0.622982] ... generic bitmap:            000000000000003f333vm-test-run-nixbot-gitlab> gitlab # [    0.623982] ... fixed-purpose counters:    0334vm-test-run-nixbot-gitlab> gitlab # [    0.624982] ... fixed-purpose bitmap:      0000000000000000335vm-test-run-nixbot-gitlab> gitlab # [    0.625982] ... value mask:                0000ffffffffffff336vm-test-run-nixbot-gitlab> gitlab # [    0.626982] ... max period:                00007fffffffffff337vm-test-run-nixbot-gitlab> gitlab # [    0.627982] ... global_ctrl mask:          000000000000003f338vm-test-run-nixbot-gitlab> gitlab # [    0.629109] signal: max sigframe size: 3376339vm-test-run-nixbot-gitlab> gitlab # [    0.630068] rcu: Hierarchical SRCU implementation.340vm-test-run-nixbot-gitlab> gitlab # [    0.630986] rcu: 	Max phase no-delay instances is 400.341vm-test-run-nixbot-gitlab> gitlab # [    0.632203] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level342vm-test-run-nixbot-gitlab> gitlab # [    0.637559] smp: Bringing up secondary CPUs ...343vm-test-run-nixbot-gitlab> gitlab # [    0.638418] smpboot: x86: Booting SMP configuration:344vm-test-run-nixbot-gitlab> gitlab # [    0.638985] .... node  #0, CPUs:      #1 #2 #3345vm-test-run-nixbot-gitlab> gitlab # [    0.643129] smp: Brought up 1 node, 4 CPUs346vm-test-run-nixbot-gitlab> gitlab # [    0.644982] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)347vm-test-run-nixbot-gitlab> gitlab # [    0.646620] Memory: 6013376K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved)348vm-test-run-nixbot-gitlab> gitlab # [    0.648981] devtmpfs: initialized349vm-test-run-nixbot-gitlab> gitlab # [    0.650200] x86/mm: Memory block size: 128MB350vm-test-run-nixbot-gitlab> gitlab # [    0.654045] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns351vm-test-run-nixbot-gitlab> gitlab # [    0.655039] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)352vm-test-run-nixbot-gitlab> gitlab # [    0.656064] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).353vm-test-run-nixbot-gitlab> gitlab # [    0.657089] pinctrl core: initialized pinctrl subsystem354vm-test-run-nixbot-gitlab> gitlab # [    0.658423] PM: RTC time: 13:29:35, date: 2026-07-04355vm-test-run-nixbot-gitlab> gitlab # [    0.662151] NET: Registered PF_NETLINK/PF_ROUTE protocol family356vm-test-run-nixbot-gitlab> gitlab # [    0.664417] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations357vm-test-run-nixbot-gitlab> gitlab # [    0.665051] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations358vm-test-run-nixbot-gitlab> gitlab # [    0.667131] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations359vm-test-run-nixbot-gitlab> gitlab # [    0.667995] audit: initializing netlink subsys (disabled)360vm-test-run-nixbot-gitlab> gitlab # [    0.669188] audit: type=2000 audit(1783171775.725:1): state=initialized audit_enabled=0 res=1361vm-test-run-nixbot-gitlab> gitlab # [    0.669324] thermal_sys: Registered thermal governor 'fair_share'362vm-test-run-nixbot-gitlab> gitlab # [    0.670982] thermal_sys: Registered thermal governor 'bang_bang'363vm-test-run-nixbot-gitlab> gitlab # [    0.671982] thermal_sys: Registered thermal governor 'step_wise'364vm-test-run-nixbot-gitlab> gitlab # [    0.672982] thermal_sys: Registered thermal governor 'user_space'365vm-test-run-nixbot-gitlab> gitlab # [    0.673982] thermal_sys: Registered thermal governor 'power_allocator'366vm-test-run-nixbot-gitlab> gitlab # [    0.675001] cpuidle: using governor menu367vm-test-run-nixbot-gitlab> gitlab # [    0.678213] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5368vm-test-run-nixbot-gitlab> gitlab # [    0.679291] PCI: Using configuration type 1 for base access369vm-test-run-nixbot-gitlab> gitlab # [    0.679982] PCI: Using configuration type 1 for extended access370vm-test-run-nixbot-gitlab> gitlab # [    0.681179] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.371vm-test-run-nixbot-gitlab> gitlab # [    0.684038] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages372vm-test-run-nixbot-gitlab> gitlab # [    0.684982] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page373vm-test-run-nixbot-gitlab> gitlab # [    0.685983] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages374vm-test-run-nixbot-gitlab> gitlab # [    0.686982] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page375vm-test-run-nixbot-gitlab> gitlab # [    0.692751] ACPI: Added _OSI(Module Device)376vm-test-run-nixbot-gitlab> gitlab # [    0.695991] ACPI: Added _OSI(Processor Device)377vm-test-run-nixbot-gitlab> gitlab # [    0.698991] ACPI: Added _OSI(Processor Aggregator Device)378vm-test-run-nixbot-gitlab> gitlab # [    0.707927] ACPI: 1 ACPI AML tables successfully acquired and loaded379vm-test-run-nixbot-gitlab> gitlab # [    0.716296] ACPI: Interpreter enabled380vm-test-run-nixbot-gitlab> gitlab # [    0.717928] ACPI: PM: (supports S0 S3 S4 S5)381vm-test-run-nixbot-gitlab> gitlab # [    0.718983] ACPI: Using IOAPIC for interrupt routing382vm-test-run-nixbot-gitlab> gitlab # [    0.720037] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug383vm-test-run-nixbot-gitlab> gitlab # [    0.720991] PCI: Using E820 reservations for host bridge windows384vm-test-run-nixbot-gitlab> gitlab # [    0.725639] ACPI: Enabled 2 GPEs in block 00 to 0F385vm-test-run-nixbot-gitlab> gitlab # [    0.743048] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])386vm-test-run-nixbot-gitlab> gitlab # [    0.747005] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]387vm-test-run-nixbot-gitlab> gitlab # [    0.755348] acpiphp: Slot [3] registered388vm-test-run-nixbot-gitlab> gitlab # [    0.758107] acpiphp: Slot [4] registered389vm-test-run-nixbot-gitlab> gitlab # [    0.761131] acpiphp: Slot [5] registered390vm-test-run-nixbot-gitlab> gitlab # [    0.765071] acpiphp: Slot [6] registered391vm-test-run-nixbot-gitlab> gitlab # [    0.767094] acpiphp: Slot [7] registered392vm-test-run-nixbot-gitlab> gitlab # [    0.770071] acpiphp: Slot [8] registered393vm-test-run-nixbot-gitlab> gitlab # [    0.772067] acpiphp: Slot [9] registered394vm-test-run-nixbot-gitlab> gitlab # [    0.775071] acpiphp: Slot [10] registered395vm-test-run-nixbot-gitlab> gitlab # [    0.777115] acpiphp: Slot [11] registered396vm-test-run-nixbot-gitlab> gitlab # [    0.780067] acpiphp: Slot [12] registered397vm-test-run-nixbot-gitlab> gitlab # [    0.782033] acpiphp: Slot [13] registered398vm-test-run-nixbot-gitlab> gitlab # [    0.784033] acpiphp: Slot [14] registered399vm-test-run-nixbot-gitlab> gitlab # [    0.785015] acpiphp: Slot [15] registered400vm-test-run-nixbot-gitlab> gitlab # [    0.786016] acpiphp: Slot [16] registered401vm-test-run-nixbot-gitlab> gitlab # [    0.788020] acpiphp: Slot [17] registered402vm-test-run-nixbot-gitlab> gitlab # [    0.789032] acpiphp: Slot [18] registered403vm-test-run-nixbot-gitlab> gitlab # [    0.790335] acpiphp: Slot [19] registered404vm-test-run-nixbot-gitlab> gitlab # [    0.792033] acpiphp: Slot [20] registered405vm-test-run-nixbot-gitlab> gitlab # [    0.793062] acpiphp: Slot [21] registered406vm-test-run-nixbot-gitlab> gitlab # [    0.796122] acpiphp: Slot [22] registered407vm-test-run-nixbot-gitlab> gitlab # [    0.800077] acpiphp: Slot [23] registered408vm-test-run-nixbot-gitlab> gitlab # [    0.803078] acpiphp: Slot [24] registered409vm-test-run-nixbot-gitlab> gitlab # [    0.806078] acpiphp: Slot [25] registered410vm-test-run-nixbot-gitlab> gitlab # [    0.809110] acpiphp: Slot [26] registered411vm-test-run-nixbot-gitlab> gitlab # [    0.812076] acpiphp: Slot [27] registered412vm-test-run-nixbot-gitlab> gitlab # [    0.816082] acpiphp: Slot [28] registered413vm-test-run-nixbot-gitlab> gitlab # [    0.819091] acpiphp: Slot [29] registered414vm-test-run-nixbot-gitlab> gitlab # [    0.822124] acpiphp: Slot [30] registered415vm-test-run-nixbot-gitlab> gitlab # [    0.826077] acpiphp: Slot [31] registered416vm-test-run-nixbot-gitlab> gitlab # [    0.829288] PCI host bridge to bus 0000:00417vm-test-run-nixbot-gitlab> gitlab # [    0.832049] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]418vm-test-run-nixbot-gitlab> gitlab # [    0.835990] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]419vm-test-run-nixbot-gitlab> gitlab # [    0.839990] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]420vm-test-run-nixbot-gitlab> gitlab # [    0.844989] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]421vm-test-run-nixbot-gitlab> gitlab # [    0.848990] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]422vm-test-run-nixbot-gitlab> gitlab # [    0.852010] pci_bus 0000:00: root bus resource [bus 00-ff]423vm-test-run-nixbot-gitlab> gitlab # [    0.854495] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint424vm-test-run-nixbot-gitlab> gitlab # [    0.859554] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint425vm-test-run-nixbot-gitlab> gitlab # [    0.868326] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint426vm-test-run-nixbot-gitlab> gitlab # [    0.879186] pci 0000:00:01.1: BAR 4 [io  0xc220-0xc22f]427vm-test-run-nixbot-gitlab> gitlab # [    0.883188] pci 0000:00:01.1: BAR 0 [io  0x01f0-0x01f7]: legacy IDE quirk428vm-test-run-nixbot-gitlab> gitlab # [    0.888991] pci 0000:00:01.1: BAR 1 [io  0x03f6]: legacy IDE quirk429vm-test-run-nixbot-gitlab> gitlab # [    0.892992] pci 0000:00:01.1: BAR 2 [io  0x0170-0x0177]: legacy IDE quirk430vm-test-run-nixbot-gitlab> gitlab # [    0.897992] pci 0000:00:01.1: BAR 3 [io  0x0376]: legacy IDE quirk431vm-test-run-nixbot-gitlab> gitlab # [    0.903785] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint432vm-test-run-nixbot-gitlab> gitlab # [    0.913121] pci 0000:00:01.2: BAR 4 [io  0xc140-0xc15f]433vm-test-run-nixbot-gitlab> gitlab # [    0.918727] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint434vm-test-run-nixbot-gitlab> gitlab # [    0.922658] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI435vm-test-run-nixbot-gitlab> gitlab # [    0.926018] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB436vm-test-run-nixbot-gitlab> gitlab # [    0.930671] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint437vm-test-run-nixbot-gitlab> gitlab # [    0.943097] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]438vm-test-run-nixbot-gitlab> gitlab # [    0.946015] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]439vm-test-run-nixbot-gitlab> gitlab # [    0.948043] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]440vm-test-run-nixbot-gitlab> gitlab # [    0.950500] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]441vm-test-run-nixbot-gitlab> gitlab # [    0.966991] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 16601 usecs442vm-test-run-nixbot-gitlab> gitlab # [    0.972092] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint443vm-test-run-nixbot-gitlab> gitlab # [    0.984031] pci 0000:00:03.0: BAR 0 [io  0xc160-0xc17f]444vm-test-run-nixbot-gitlab> gitlab # [    0.989029] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]445vm-test-run-nixbot-gitlab> gitlab # [    0.993124] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]446vm-test-run-nixbot-gitlab> gitlab # [    0.999045] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]447vm-test-run-nixbot-gitlab> gitlab # [    1.010349] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint448vm-test-run-nixbot-gitlab> gitlab # [    1.024029] pci 0000:00:04.0: BAR 0 [io  0xc180-0xc19f]449vm-test-run-nixbot-gitlab> gitlab # [    1.027017] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]450vm-test-run-nixbot-gitlab> gitlab # [    1.031115] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]451vm-test-run-nixbot-gitlab> gitlab # [    1.039653] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint452vm-test-run-nixbot-gitlab> gitlab # [    1.050027] pci 0000:00:05.0: BAR 0 [io  0xc1a0-0xc1bf]453vm-test-run-nixbot-gitlab> gitlab # [    1.054018] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]454vm-test-run-nixbot-gitlab> gitlab # [    1.057095] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]455vm-test-run-nixbot-gitlab> gitlab # [    1.066258] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint456vm-test-run-nixbot-gitlab> gitlab # [    1.078020] pci 0000:00:06.0: BAR 0 [io  0xc1c0-0xc1df]457vm-test-run-nixbot-gitlab> gitlab # [    1.081040] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]458vm-test-run-nixbot-gitlab> gitlab # [    1.086126] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]459vm-test-run-nixbot-gitlab> gitlab # [    1.097787] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint460vm-test-run-nixbot-gitlab> gitlab # [    1.111016] pci 0000:00:07.0: BAR 0 [io  0xc000-0xc07f]461vm-test-run-nixbot-gitlab> gitlab # [    1.114017] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]462vm-test-run-nixbot-gitlab> gitlab # [    1.118094] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]463vm-test-run-nixbot-gitlab> gitlab # [    1.127281] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint464vm-test-run-nixbot-gitlab> gitlab # [    1.135024] pci 0000:00:08.0: BAR 0 [io  0xc080-0xc0ff]465vm-test-run-nixbot-gitlab> gitlab # [    1.139026] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]466vm-test-run-nixbot-gitlab> gitlab # [    1.144142] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]467vm-test-run-nixbot-gitlab> gitlab # [    1.156358] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint468vm-test-run-nixbot-gitlab> gitlab # [    1.171032] pci 0000:00:09.0: BAR 0 [io  0xc1e0-0xc1ff]469vm-test-run-nixbot-gitlab> gitlab # [    1.176029] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]470vm-test-run-nixbot-gitlab> gitlab # [    1.180124] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]471vm-test-run-nixbot-gitlab> gitlab # [    1.185018] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]472vm-test-run-nixbot-gitlab> gitlab # [    1.193202] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint473vm-test-run-nixbot-gitlab> gitlab # [    1.204070] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]474vm-test-run-nixbot-gitlab> gitlab # [    1.208144] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]475vm-test-run-nixbot-gitlab> gitlab # [    1.220477] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint476vm-test-run-nixbot-gitlab> gitlab # [    1.233031] pci 0000:00:0b.0: BAR 0 [io  0xc100-0xc13f]477vm-test-run-nixbot-gitlab> gitlab # [    1.237029] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]478vm-test-run-nixbot-gitlab> gitlab # [    1.242151] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]479vm-test-run-nixbot-gitlab> gitlab # [    1.252461] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint480vm-test-run-nixbot-gitlab> gitlab # [    1.264024] pci 0000:00:0c.0: BAR 0 [io  0xc200-0xc21f]481vm-test-run-nixbot-gitlab> gitlab # [    1.267045] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]482vm-test-run-nixbot-gitlab> gitlab # [    1.269404] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]483vm-test-run-nixbot-gitlab> gitlab # [    1.279897] ACPI: PCI: Interrupt link LNKA configured for IRQ 10484vm-test-run-nixbot-gitlab> gitlab # [    1.282211] ACPI: PCI: Interrupt link LNKB configured for IRQ 10485vm-test-run-nixbot-gitlab> gitlab # [    1.284193] ACPI: PCI: Interrupt link LNKC configured for IRQ 11486vm-test-run-nixbot-gitlab> gitlab # [    1.286179] ACPI: PCI: Interrupt link LNKD configured for IRQ 11487vm-test-run-nixbot-gitlab> gitlab # [    1.288098] ACPI: PCI: Interrupt link LNKS configured for IRQ 9488vm-test-run-nixbot-gitlab> gitlab # [    1.292140] iommu: Default domain type: Translated489vm-test-run-nixbot-gitlab> gitlab # [    1.292982] iommu: DMA domain TLB invalidation policy: lazy mode490vm-test-run-nixbot-gitlab> gitlab # [    1.295326] ACPI: bus type USB registered491vm-test-run-nixbot-gitlab> gitlab # [    1.297090] usbcore: registered new interface driver usbfs492vm-test-run-nixbot-gitlab> gitlab # [    1.299023] usbcore: registered new interface driver hub493vm-test-run-nixbot-gitlab> gitlab # [    1.301005] usbcore: registered new device driver usb494vm-test-run-nixbot-gitlab> gitlab # [    1.303421] NetLabel: Initializing495vm-test-run-nixbot-gitlab> gitlab # [    1.304982] NetLabel:  domain hash size = 128496vm-test-run-nixbot-gitlab> gitlab # [    1.305982] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO497vm-test-run-nixbot-gitlab> gitlab # [    1.308089] NetLabel:  unlabeled traffic allowed by default498vm-test-run-nixbot-gitlab> gitlab # [    1.308997] PCI: Using ACPI for IRQ routing499vm-test-run-nixbot-gitlab> gitlab # [    1.312021] pci 0000:00:02.0: vgaarb: setting as boot VGA device500vm-test-run-nixbot-gitlab> gitlab # [    1.312977] pci 0000:00:02.0: vgaarb: bridge control possible501vm-test-run-nixbot-gitlab> gitlab # [    1.312977] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none502vm-test-run-nixbot-gitlab> gitlab # [    1.317987] vgaarb: loaded503vm-test-run-nixbot-gitlab> gitlab # [    1.319420] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0504vm-test-run-nixbot-gitlab> gitlab # [    1.321985] hpet0: 3 comparators, 64-bit 100.000000 MHz counter505vm-test-run-nixbot-gitlab> gitlab # [    1.327981] clocksource: Switched to clocksource kvm-clock506vm-test-run-nixbot-gitlab> gitlab # [    1.332595] VFS: Disk quotas dquot_6.6.0507vm-test-run-nixbot-gitlab> gitlab # [    1.333891] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)508vm-test-run-nixbot-gitlab> gitlab # [    1.336264] pnp: PnP ACPI init509vm-test-run-nixbot-gitlab> gitlab # [    1.338034] pnp: PnP ACPI: found 6 devices510vm-test-run-nixbot-gitlab> gitlab # [    1.348857] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns511vm-test-run-nixbot-gitlab> gitlab # [    1.351591] clocksource: Switched to clocksource acpi_pm512vm-test-run-nixbot-gitlab> gitlab # [    1.353516] NET: Registered PF_INET protocol family513vm-test-run-nixbot-gitlab> gitlab # [    1.359397] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)514vm-test-run-nixbot-gitlab> gitlab # [    1.384147] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)515vm-test-run-nixbot-gitlab> gitlab # [    1.386802] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)516vm-test-run-nixbot-gitlab> gitlab # [    1.389306] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)517vm-test-run-nixbot-gitlab> gitlab # [    1.394459] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)518vm-test-run-nixbot-gitlab> gitlab # [    1.396881] TCP: Hash tables configured (established 65536 bind 65536)519vm-test-run-nixbot-gitlab> gitlab # [    1.399292] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)520vm-test-run-nixbot-gitlab> gitlab # [    1.401990] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)521vm-test-run-nixbot-gitlab> gitlab # [    1.404318] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)522vm-test-run-nixbot-gitlab> gitlab # [    1.406767] NET: Registered PF_UNIX/PF_LOCAL protocol family523vm-test-run-nixbot-gitlab> gitlab # [    1.408602] NET: Registered PF_XDP protocol family524vm-test-run-nixbot-gitlab> gitlab # [    1.410230] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]525vm-test-run-nixbot-gitlab> gitlab # [    1.412133] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]526vm-test-run-nixbot-gitlab> gitlab # [    1.414015] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]527vm-test-run-nixbot-gitlab> gitlab # [    1.416237] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]528vm-test-run-nixbot-gitlab> gitlab # [    1.421616] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]529vm-test-run-nixbot-gitlab> gitlab # [    1.427629] pci 0000:00:01.0: PIIX3: Enabling Passive Release530vm-test-run-nixbot-gitlab> gitlab # [    1.432136] pci 0000:00:00.0: Limiting direct PCI/PCI transfers531vm-test-run-nixbot-gitlab> gitlab # [    1.441802] ACPI: \_SB_.LNKD: Enabled at IRQ 11532vm-test-run-nixbot-gitlab> gitlab # [    1.450298] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 12994 usecs533vm-test-run-nixbot-gitlab> gitlab # [    1.456622] PCI: CLS 0 bytes, default 64534vm-test-run-nixbot-gitlab> gitlab # [    1.460232] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)535vm-test-run-nixbot-gitlab> gitlab # [    1.460496] Trying to unpack rootfs image as initramfs...536vm-test-run-nixbot-gitlab> gitlab # [    1.461148] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB)537vm-test-run-nixbot-gitlab> gitlab # [    1.473926] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns538vm-test-run-nixbot-gitlab> gitlab # [    1.522931] Initialise system trusted keyrings539vm-test-run-nixbot-gitlab> gitlab # [    1.527479] workingset: timestamp_bits=40 max_order=21 bucket_order=0540vm-test-run-nixbot-gitlab> gitlab # [    1.555797] Key type asymmetric registered541vm-test-run-nixbot-gitlab> gitlab # [    1.557141] Asymmetric key parser 'x509' registered542vm-test-run-nixbot-gitlab> gitlab # [    1.558777] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)543vm-test-run-nixbot-gitlab> gitlab # [    1.561433] io scheduler mq-deadline registered544vm-test-run-nixbot-gitlab> gitlab # [    1.562927] io scheduler kyber registered545vm-test-run-nixbot-gitlab> gitlab # [    1.567305] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled546vm-test-run-nixbot-gitlab> gitlab # [    1.569556] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A547vm-test-run-nixbot-gitlab> gitlab # [    1.574303] Linux agpgart interface v0.103548vm-test-run-nixbot-gitlab> gitlab # [    1.575767] ACPI: bus type drm_connector registered549vm-test-run-nixbot-gitlab> gitlab # [    1.579505] usbcore: registered new interface driver usbserial_generic550vm-test-run-nixbot-gitlab> gitlab # [    1.581515] usbserial: USB Serial support registered for generic551vm-test-run-nixbot-gitlab> gitlab # [    1.583380] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.552vm-test-run-nixbot-gitlab> gitlab # [    1.583380] Please enable it if your BIOS has the CPPC option.553vm-test-run-nixbot-gitlab> gitlab # [    1.587438] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled554vm-test-run-nixbot-gitlab> gitlab # [    1.589904] drop_monitor: Initializing network drop monitor service555vm-test-run-nixbot-gitlab> gitlab # [    1.591999] NET: Registered PF_INET6 protocol family556vm-test-run-nixbot-gitlab> gitlab # [    1.594585] Segment Routing with IPv6557vm-test-run-nixbot-gitlab> gitlab # [    1.595860] In-situ OAM (IOAM) with IPv6558vm-test-run-nixbot-gitlab> gitlab # [    1.599151] IPI shorthand broadcast: enabled559vm-test-run-nixbot-gitlab> gitlab # [    1.609212] sched_clock: Marking stable (1345066367, 263644326)->(1698226577, -89515884)560vm-test-run-nixbot-gitlab> gitlab # [    1.612252] registered taskstats version 1561vm-test-run-nixbot-gitlab> gitlab # [    1.614025] Loading compiled-in X.509 certificates562vm-test-run-nixbot-gitlab> gitlab # [    1.636201] Demotion targets for Node 0: null563vm-test-run-nixbot-gitlab> gitlab # [    1.639724] Key type .fscrypt registered564vm-test-run-nixbot-gitlab> gitlab # [    1.642192] Key type fscrypt-provisioning registered565vm-test-run-nixbot-gitlab> gitlab # [    1.645534] ima: No TPM chip found, activating TPM-bypass!566vm-test-run-nixbot-gitlab> gitlab # [    1.648850] ima: Allocated hash algorithm: sha1567vm-test-run-nixbot-gitlab> gitlab # [    1.651797] ima: No architecture policies found568vm-test-run-nixbot-gitlab> gitlab # [    1.655349] PM:   Magic number: 2:610:482569vm-test-run-nixbot-gitlab> gitlab # [    1.657871] RAS: Correctable Errors collector initialized.570vm-test-run-nixbot-gitlab> gitlab # [    1.663354] clk: Disabling unused clocks571vm-test-run-nixbot-gitlab> gitlab # [    1.664678] PM: genpd: Disabling unused power domains572vm-test-run-nixbot-gitlab> gitlab # [    1.760264] Freeing initrd memory: 27584K573vm-test-run-nixbot-gitlab> gitlab # [    1.766934] Freeing unused decrypted memory: 2028K574vm-test-run-nixbot-gitlab> gitlab # [    1.773841] Freeing unused kernel image (initmem) memory: 3644K575vm-test-run-nixbot-gitlab> gitlab # [    1.777535] Write protecting the kernel read-only data: 32768k576vm-test-run-nixbot-gitlab> gitlab # [    1.784853] Freeing unused kernel image (text/rodata gap) memory: 1332K577vm-test-run-nixbot-gitlab> gitlab # [    1.791346] Freeing unused kernel image (rodata/data gap) memory: 816K578vm-test-run-nixbot-gitlab> gitlab # [    1.856242] x86/mm: Checked W+X mappings: passed, no W+X pages found.579vm-test-run-nixbot-gitlab> gitlab # [    1.858246] Run /init as init process580vm-test-run-nixbot-gitlab> gitlab # [    1.874560] systemd[1]: Inserted module 'autofs4'581vm-test-run-nixbot-gitlab> gitlab # [    1.903264] fuse: init (API version 7.45)582vm-test-run-nixbot-gitlab> gitlab # [    1.914015] ACPI: \_SB_.LNKC: Enabled at IRQ 10583vm-test-run-nixbot-gitlab> gitlab # [    1.935877] ACPI: \_SB_.LNKA: Enabled at IRQ 10584vm-test-run-nixbot-gitlab> gitlab # [    1.940905] ACPI: \_SB_.LNKB: Enabled at IRQ 11585vm-test-run-nixbot-gitlab> gitlab # [    1.992839] systemd[1]: Successfully made /usr/ read-only.586vm-test-run-nixbot-gitlab> gitlab # [    2.333300] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)587vm-test-run-nixbot-gitlab> gitlab # [    2.355154] systemd[1]: Detected virtualization kvm.588vm-test-run-nixbot-gitlab> gitlab # [    2.359172] systemd[1]: Detected architecture x86-64.589vm-test-run-nixbot-gitlab> gitlab # [    2.363198] systemd[1]: Running in initrd.590vm-test-run-nixbot-gitlab> gitlab # [    2.367587] systemd[1]: Initializing machine ID from random generator.591vm-test-run-nixbot-gitlab> gitlab # [    2.373042] systemd[1]: Hostname set to <gitlab>.592vm-test-run-nixbot-gitlab> gitlab # [    2.456616] systemd[1]: Queued start job for default target Initrd Default Target.593vm-test-run-nixbot-gitlab> gitlab # [    2.629085] systemd[1]: Created slice Slice /system/modprobe.594vm-test-run-nixbot-gitlab> gitlab # [    2.634416] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.595vm-test-run-nixbot-gitlab> gitlab # [    2.640636] systemd[1]: Expecting device /dev/disk/by-label/nix-store...596vm-test-run-nixbot-gitlab> gitlab # [    2.645868] systemd[1]: Expecting device /dev/disk/by-label/nixos...597vm-test-run-nixbot-gitlab> gitlab # [    2.650937] systemd[1]: Reached target Path Units.598vm-test-run-nixbot-gitlab> gitlab # [    2.654988] systemd[1]: Reached target Slice Units.599vm-test-run-nixbot-gitlab> gitlab # [    2.659088] systemd[1]: Reached target Swaps.600vm-test-run-nixbot-gitlab> gitlab # [    2.662921] systemd[1]: Reached target Timer Units.601vm-test-run-nixbot-gitlab> gitlab # [    2.667331] systemd[1]: Listening on D-Bus System Message Bus Socket.602vm-test-run-nixbot-gitlab> gitlab # [    2.671999] systemd[1]: Listening on Journal Socket (/dev/log).603vm-test-run-nixbot-gitlab> gitlab # [    2.676105] systemd[1]: Listening on Journal Sockets.604vm-test-run-nixbot-gitlab> gitlab # [    2.679890] systemd[1]: Listening on udev Control Socket.605vm-test-run-nixbot-gitlab> gitlab # [    2.683480] systemd[1]: Listening on udev Kernel Socket.606vm-test-run-nixbot-gitlab> gitlab # [    2.686297] systemd[1]: Reached target Socket Units.607vm-test-run-nixbot-gitlab> gitlab # [    2.692988] systemd[1]: Starting Create List of Static Device Nodes...608vm-test-run-nixbot-gitlab> gitlab # [    2.699223] systemd[1]: Starting Load Kernel Module 9pnet_virtio...609vm-test-run-nixbot-gitlab> gitlab # [    2.719357] systemd[1]: Starting Load Kernel Module configfs...610vm-test-run-nixbot-gitlab> gitlab # [    2.725170] systemd[1]: Starting Journal Service...611vm-test-run-nixbot-gitlab> gitlab # [    2.732278] systemd[1]: Starting Load Kernel Modules...612vm-test-run-nixbot-gitlab> gitlab # [    2.734006] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki613vm-test-run-nixbot-gitlab> gitlab # [    2.749847] systemd[1]: Starting Coldplug All udev Devices...614vm-test-run-nixbot-gitlab> gitlab # [    2.771262] netfs: FS-Cache loaded615vm-test-run-nixbot-gitlab> gitlab # [    2.778611] 9pnet: Installing 9P2000 support616vm-test-run-nixbot-gitlab> gitlab # [    2.781949] systemd[1]: Finished Create List of Static Device Nodes.617vm-test-run-nixbot-gitlab> gitlab # [    2.786601] systemd-journald[151]: Collecting audit messages is disabled.618vm-test-run-nixbot-gitlab> gitlab # [    2.793848] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.619vm-test-run-nixbot-gitlab> gitlab # [    2.793865] systemd[1]: modprobe@configfs.service: Deactivated successfully.620vm-test-run-nixbot-gitlab> gitlab # [    2.794550] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev621vm-test-run-nixbot-gitlab> gitlab # [    2.803494] systemd[1]: Finished Load Kernel Module configfs.622vm-test-run-nixbot-gitlab> gitlab # [    2.810098] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.623vm-test-run-nixbot-gitlab> gitlab # [    2.814267] systemd[1]: Finished Load Kernel Module 9pnet_virtio.624vm-test-run-nixbot-gitlab> gitlab # [    2.820847] systemd[1]: Finished Load Kernel Modules.625vm-test-run-nixbot-gitlab> gitlab # [    2.823406] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config626vm-test-run-nixbot-gitlab> gitlab # [    2.829123] systemd[1]: Starting Apply Kernel Variables...627vm-test-run-nixbot-gitlab> gitlab # [    2.841011] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...628vm-test-run-nixbot-gitlab> gitlab # [    2.851440] systemd[1]: Finished Apply Kernel Variables.629vm-test-run-nixbot-gitlab> gitlab # [    2.855482] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.630vm-test-run-nixbot-gitlab> gitlab # [    2.860454] systemd[1]: Starting Create Static Device Nodes in /dev...631vm-test-run-nixbot-gitlab> gitlab # [    2.871067] systemd[1]: Started Journal Service.632vm-test-run-nixbot-gitlab> gitlab # [    2.608420] systemd-modules-load[154]: Using 4 probe threads633vm-test-run-nixbot-gitlab> gitlab # [    2.611510] systemd-modules-load[154]: Inserted module 'virtio_balloon'634vm-test-run-nixbot-gitlab> gitlab # [    2.613589] systemd-modules-load[154]: Inserted module 'dm_mod'635vm-test-run-nixbot-gitlab> gitlab # [    2.615550] systemd-modules-load[154]: Inserted module 'virtio_gpu'636vm-test-run-nixbot-gitlab> gitlab # [    2.617528] systemd[1]: Finished Create Static Device Nodes in /dev.637vm-test-run-nixbot-gitlab> gitlab # [    2.619745] systemd[1]: Reached target Preparation for Local File Systems.638vm-test-run-nixbot-gitlab> gitlab # [    2.621862] systemd[1]: Reached target Local File Systems.639vm-test-run-nixbot-gitlab> gitlab # [    2.623673] systemd[1]: Starting Create System Files and Directories...640vm-test-run-nixbot-gitlab> gitlab # [    2.625644] systemd[1]: Starting Rule-based Manager for Device Events and Files...641vm-test-run-nixbot-gitlab> gitlab # [    2.637278] systemd[1]: Finished Create System Files and Directories.642vm-test-run-nixbot-gitlab> gitlab # [    2.650286] systemd-udevd[189]: Using default interface naming scheme 'v260'.643vm-test-run-nixbot-gitlab> gitlab # [    2.666792] systemd[1]: Started Rule-based Manager for Device Events and Files.644vm-test-run-nixbot-gitlab> gitlab # [    2.679302] systemd[1]: Finished Coldplug All udev Devices.645vm-test-run-nixbot-gitlab> gitlab # [    2.680952] systemd[1]: Reached target System Initialization.646vm-test-run-nixbot-gitlab> gitlab # [    2.683293] systemd[1]: Reached target Basic System.647vm-test-run-nixbot-gitlab> gitlab # [    3.113974] uhci_hcd 0000:00:01.2: UHCI Host Controller648vm-test-run-nixbot-gitlab> gitlab # [    3.115335] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1649vm-test-run-nixbot-gitlab> gitlab # [    3.121044] uhci_hcd 0000:00:01.2: detected 2 ports650vm-test-run-nixbot-gitlab> gitlab # [    3.123503] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12651vm-test-run-nixbot-gitlab> gitlab # [    3.124033] SCSI subsystem initialized652vm-test-run-nixbot-gitlab> gitlab # [    3.125038] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140653vm-test-run-nixbot-gitlab> gitlab # [    3.125439] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18654vm-test-run-nixbot-gitlab> gitlab # [    3.125443] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1655vm-test-run-nixbot-gitlab> gitlab # [    3.125445] usb usb1: Product: UHCI Host Controller656vm-test-run-nixbot-gitlab> gitlab # [    3.125446] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd657vm-test-run-nixbot-gitlab> gitlab # [    3.125447] usb usb1: SerialNumber: 0000:00:01.2658vm-test-run-nixbot-gitlab> gitlab # [    3.127865] hub 1-0:1.0: USB hub found659vm-test-run-nixbot-gitlab> gitlab # [    3.132579] serio: i8042 KBD port at 0x60,0x64 irq 1660vm-test-run-nixbot-gitlab> gitlab # [    3.134954] hub 1-0:1.0: 2 ports detected661vm-test-run-nixbot-gitlab> gitlab # [    3.141733] serio: i8042 AUX port at 0x60,0x64 irq 12662vm-test-run-nixbot-gitlab> gitlab # [    3.151313] virtio_blk virtio4: 4/0/0 default/read/poll queues663vm-test-run-nixbot-gitlab> gitlab # [    3.157762] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)664vm-test-run-nixbot-gitlab> gitlab # [    3.163204] virtio_blk virtio5: 4/0/0 default/read/poll queues665vm-test-run-nixbot-gitlab> gitlab # [    3.173116] virtio_blk virtio5: [vdb] 11450032 512-byte logical blocks (5.86 GB/5.46 GiB)666vm-test-run-nixbot-gitlab> gitlab # [    3.187713] scsi host0: ata_piix667vm-test-run-nixbot-gitlab> gitlab # [    3.189595] scsi host1: ata_piix668vm-test-run-nixbot-gitlab> gitlab # [    3.190958] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0669vm-test-run-nixbot-gitlab> gitlab # [    3.193479] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0670vm-test-run-nixbot-gitlab> gitlab # [    2.945246] systemd[1]: Starting Virtual Console Setup...671vm-test-run-nixbot-gitlab> gitlab # [    2.962502] systemd[1]: Found device /dev/disk/by-label/nix-store.672vm-test-run-nixbot-gitlab> gitlab # [    2.967448] systemd[1]: Found device /dev/disk/by-label/nixos.673vm-test-run-nixbot-gitlab> gitlab # [    2.969170] systemd[1]: Reached target Initrd Root Device.674vm-test-run-nixbot-gitlab> gitlab # [    2.972981] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...675vm-test-run-nixbot-gitlab> gitlab # [    2.978614] systemd-vconsole-setup[232]: Configuration of first virtual console was skipped, ignoring remaining ones.676vm-test-run-nixbot-gitlab> gitlab # [    2.991950] systemd[1]: Finished Virtual Console Setup.677vm-test-run-nixbot-gitlab> gitlab # [    3.007874] systemd-fsck[239]: nixos: clean, 12/65536 files, 13019/262144 blocks678vm-test-run-nixbot-gitlab> gitlab # [    3.014057] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.679vm-test-run-nixbot-gitlab> gitlab # [    3.347795] ata2: found unknown device (class 0)680vm-test-run-nixbot-gitlab> gitlab # [    3.349845] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100681vm-test-run-nixbot-gitlab> gitlab # [    3.355967] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5682vm-test-run-nixbot-gitlab> gitlab # [    3.363018] usb 1-1: new full-speed USB device number 2 using uhci_hcd683vm-test-run-nixbot-gitlab> gitlab # [    3.404771] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0684vm-test-run-nixbot-gitlab> gitlab # [    3.142787] (udev-worker)[218]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.685vm-test-run-nixbot-gitlab> gitlab # [    3.157309] (udev-worker)[204]: Network interface NamePolicy= disabled on kernel command line.686vm-test-run-nixbot-gitlab> gitlab # [    3.163597] (udev-worker)[218]: Network interface NamePolicy= disabled on kernel command line.687vm-test-run-nixbot-gitlab> gitlab # [    3.446759] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray688vm-test-run-nixbot-gitlab> gitlab # [    3.474090] cdrom: Uniform CD-ROM driver Revision: 3.20689vm-test-run-nixbot-gitlab> gitlab # [    3.535967] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00690vm-test-run-nixbot-gitlab> gitlab # [    3.540831] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10691vm-test-run-nixbot-gitlab> gitlab # [    3.545903] usb 1-1: Product: QEMU USB Tablet692vm-test-run-nixbot-gitlab> gitlab # [    3.548989] usb 1-1: Manufacturer: QEMU693vm-test-run-nixbot-gitlab> gitlab # [    3.551594] usb 1-1: SerialNumber: 28754-0000:00:01.2-1694vm-test-run-nixbot-gitlab> gitlab # [    3.591967] hid: raw HID events driver (C) Jiri Kosina695vm-test-run-nixbot-gitlab> gitlab # [    3.605837] usbcore: registered new interface driver usbhid696vm-test-run-nixbot-gitlab> gitlab # [    3.608716] usbhid: USB HID core driver697vm-test-run-nixbot-gitlab> gitlab # [    3.619824] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2698vm-test-run-nixbot-gitlab> gitlab # [    3.623174] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0699vm-test-run-nixbot-gitlab> gitlab # [    3.513810] systemd[1]: Mounting /sysroot...700vm-test-run-nixbot-gitlab> gitlab # [    3.956289] EXT4-fs (vda): mounted filesystem 393e42d7-bbea-4364-93d9-c419e048f38d r/w with ordered data mode. Quota mode: none.701vm-test-run-nixbot-gitlab> gitlab # [    3.698633] systemd[1]: Mounted /sysroot.702vm-test-run-nixbot-gitlab> gitlab # [    3.700372] systemd[1]: Reached target Initrd Root File System.703vm-test-run-nixbot-gitlab> gitlab # [    3.708561] systemd[1]: Mounting /sysroot/nix/.ro-store...704vm-test-run-nixbot-gitlab> gitlab # [    3.713317] systemd[1]: Mounting /sysroot/nix/.rw-store...705vm-test-run-nixbot-gitlab> gitlab # [    3.718251] systemd[1]: Mounting /sysroot/run...706vm-test-run-nixbot-gitlab> gitlab # [    3.722434] systemd[1]: Mounting /sysroot/tmp/shared...707vm-test-run-nixbot-gitlab> gitlab # [    3.725931] systemd[1]: Mounting /sysroot/tmp/xchg...708vm-test-run-nixbot-gitlab> gitlab # [    3.728996] systemd[1]: Starting Mountpoints Configured in the Real Root...709vm-test-run-nixbot-gitlab> gitlab # [    3.732209] systemd[1]: Mounted /sysroot/nix/.rw-store.710vm-test-run-nixbot-gitlab> gitlab # [    4.002817] erofs (device vdb): mounted with root inode @ nid 128.711vm-test-run-nixbot-gitlab> gitlab # [    3.742400] systemd[1]: Mounted /sysroot/nix/.ro-store.712vm-test-run-nixbot-gitlab> gitlab # [    4.007557] 9p: Installing v9fs 9p2000 file system support713vm-test-run-nixbot-gitlab> gitlab # [    3.744314] systemd[1]: Mounted /sysroot/run.714vm-test-run-nixbot-gitlab> gitlab # [    3.750213] systemd[1]: Starting rw-sysroot-nix-store.service...715vm-test-run-nixbot-gitlab> gitlab # [    3.751988] systemd-sysroot-fstab-check[272]: /sysroot should be mounted in the initrd, will request daemon-reload.716vm-test-run-nixbot-gitlab> gitlab # [    3.755334] systemd[1]: Mounted /sysroot/tmp/shared.717vm-test-run-nixbot-gitlab> gitlab # [    3.756634] systemd[1]: Mounted /sysroot/tmp/xchg.718vm-test-run-nixbot-gitlab> gitlab # [    3.759767] systemd[1]: Reload requested from client PID 272 ('systemd-sysroot') (unit initrd-parse-etc.service)...719vm-test-run-nixbot-gitlab> gitlab # [    3.762540] systemd[1]: Reloading...720vm-test-run-nixbot-gitlab> gitlab # [    3.963405] systemd[1]: Reloading finished in 202 ms.721vm-test-run-nixbot-gitlab> gitlab # [    4.027652] systemd-sysroot-fstab-check[272]: Requesting initrd-fs.target/start/replace...722vm-test-run-nixbot-gitlab> gitlab # [    4.183241] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.723vm-test-run-nixbot-gitlab> gitlab # [    4.191146] systemd[1]: Finished rw-sysroot-nix-store.service.724vm-test-run-nixbot-gitlab> gitlab # [    4.205273] systemd-sysroot-fstab-check[272]: Requesting swap.target/start/replace...725vm-test-run-nixbot-gitlab> gitlab # [    4.212281] systemd[1]: Starting rw-sysroot-nix-store.service...726vm-test-run-nixbot-gitlab> gitlab # [    4.215551] systemd[1]: initrd-parse-etc.service: Deactivated successfully.727vm-test-run-nixbot-gitlab> gitlab # [    4.219167] systemd[1]: Finished Mountpoints Configured in the Real Root.728vm-test-run-nixbot-gitlab> gitlab # [    4.222761] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.729vm-test-run-nixbot-gitlab> gitlab # [    4.261273] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.730vm-test-run-nixbot-gitlab> gitlab # [    4.269785] systemd[1]: Finished rw-sysroot-nix-store.service.731vm-test-run-nixbot-gitlab> gitlab # [    4.515220] systemd[1]: Mounting /sysroot/nix/store...732vm-test-run-nixbot-gitlab> gitlab # [    4.566339] systemd[1]: Mounted /sysroot/nix/store.733vm-test-run-nixbot-gitlab> gitlab # [    4.569940] systemd[1]: Reached target Initrd File Systems.734vm-test-run-nixbot-gitlab> gitlab # [    4.574249] systemd[1]: Starting Find NixOS closure...735vm-test-run-nixbot-gitlab> gitlab # [    4.580150] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...736vm-test-run-nixbot-gitlab> gitlab # [    4.616867] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.737vm-test-run-nixbot-gitlab> gitlab # [    4.621660] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.738vm-test-run-nixbot-gitlab> gitlab # [    4.643850] systemd[1]: Finished Find NixOS closure.739vm-test-run-nixbot-gitlab> gitlab # [    4.647534] systemd[1]: Reached target Initrd Default Target.740vm-test-run-nixbot-gitlab> gitlab # [    4.655496] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...741vm-test-run-nixbot-gitlab> gitlab # [    4.701421] systemd[1]: Stopped target Initrd Default Target.742vm-test-run-nixbot-gitlab> gitlab # [    4.705168] systemd[1]: Stopped target Basic System.743vm-test-run-nixbot-gitlab> gitlab # [    4.708427] systemd[1]: Stopped target Initrd Root Device.744vm-test-run-nixbot-gitlab> gitlab # [    4.711826] systemd[1]: Stopped target Path Units.745vm-test-run-nixbot-gitlab> gitlab # [    4.714845] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.746vm-test-run-nixbot-gitlab> gitlab # [    4.719373] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.747vm-test-run-nixbot-gitlab> gitlab # [    4.723975] systemd[1]: Stopped target Slice Units.748vm-test-run-nixbot-gitlab> gitlab # [    4.727407] systemd[1]: Stopped target Socket Units.749vm-test-run-nixbot-gitlab> gitlab # [    4.730554] systemd[1]: Stopped target System Initialization.750vm-test-run-nixbot-gitlab> gitlab # [    4.734141] systemd[1]: Stopped target Swaps.751vm-test-run-nixbot-gitlab> gitlab # [    4.736986] systemd[1]: Stopped target Timer Units.752vm-test-run-nixbot-gitlab> gitlab # [    4.740133] systemd[1]: dbus.socket: Deactivated successfully.753vm-test-run-nixbot-gitlab> gitlab # [    4.743378] systemd[1]: Closed D-Bus System Message Bus Socket.754vm-test-run-nixbot-gitlab> gitlab # [    4.746289] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.755vm-test-run-nixbot-gitlab> gitlab # [    4.749797] systemd[1]: Stopped Find NixOS closure.756vm-test-run-nixbot-gitlab> gitlab # [    4.752278] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio757vm-test-run-nixbot-gitlab> gitlab # [    4.757454] systemd[1]: Starting rw-sysroot-nix-store.service...758vm-test-run-nixbot-gitlab> gitlab # [    4.759975] systemd[1]: systemd-sysctl.service: Deactivated successfully.759vm-test-run-nixbot-gitlab> gitlab # [    4.761978] systemd[1]: Stopped Apply Kernel Variables.760vm-test-run-nixbot-gitlab> gitlab # [    4.763409] systemd[1]: systemd-modules-load.service: Deactivated successfully.761vm-test-run-nixbot-gitlab> gitlab # [    4.765258] systemd[1]: Stopped Load Kernel Modules.762vm-test-run-nixbot-gitlab> gitlab # [    4.768040] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.763vm-test-run-nixbot-gitlab> gitlab # [    4.769958] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.764vm-test-run-nixbot-gitlab> gitlab # [    4.776313] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.765vm-test-run-nixbot-gitlab> gitlab # [    4.778411] systemd[1]: Stopped Create System Files and Directories.766vm-test-run-nixbot-gitlab> gitlab # [    4.782663] systemd[1]: Stopped target Local File Systems.767vm-test-run-nixbot-gitlab> gitlab # [    4.784336] systemd[1]: Stopped target Preparation for Local File Systems.768vm-test-run-nixbot-gitlab> gitlab # [    4.786322] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.769vm-test-run-nixbot-gitlab> gitlab # [    4.788402] systemd[1]: Stopped Coldplug All udev Devices.770vm-test-run-nixbot-gitlab> gitlab # [    4.791116] systemd[1]: Stopping Rule-based Manager for Device Events and Files...771vm-test-run-nixbot-gitlab> gitlab # [    4.793187] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.772vm-test-run-nixbot-gitlab> gitlab # [    4.795137] systemd[1]: Stopped Virtual Console Setup.773vm-test-run-nixbot-gitlab> gitlab # [    4.802367] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.774vm-test-run-nixbot-gitlab> gitlab # [    4.804380] systemd[1]: Finished rw-sysroot-nix-store.service.775vm-test-run-nixbot-gitlab> gitlab # [    4.808552] systemd[1]: systemd-udevd.service: Deactivated successfully.776vm-test-run-nixbot-gitlab> gitlab # [    4.811754] systemd[1]: Stopped Rule-based Manager for Device Events and Files.777vm-test-run-nixbot-gitlab> gitlab # [    4.813456] systemd[1]: systemd-udevd.service: Consumed 1.120s CPU time over 2.183s wall clock time, 24.7M memory peak.778vm-test-run-nixbot-gitlab> gitlab # [    4.820825] systemd[1]: initrd-cleanup.service: Deactivated successfully.779vm-test-run-nixbot-gitlab> gitlab # [    4.824367] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.780vm-test-run-nixbot-gitlab> gitlab # [    4.830518] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.781vm-test-run-nixbot-gitlab> gitlab # [    4.832276] systemd[1]: Closed udev Control Socket.782vm-test-run-nixbot-gitlab> gitlab # [    4.835567] systemd[1]: Starting Cleanup udev Database...783vm-test-run-nixbot-gitlab> gitlab # [    4.836915] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.784vm-test-run-nixbot-gitlab> gitlab # [    4.838784] systemd[1]: Stopped Create Static Device Nodes in /dev.785vm-test-run-nixbot-gitlab> gitlab # [    4.843081] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.786vm-test-run-nixbot-gitlab> gitlab # [    4.845050] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.787vm-test-run-nixbot-gitlab> gitlab # [    4.850716] systemd[1]: kmod-static-nodes.service: Deactivated successfully.788vm-test-run-nixbot-gitlab> gitlab # [    4.853161] systemd[1]: Stopped Create List of Static Device Nodes.789vm-test-run-nixbot-gitlab> gitlab # [    4.862524] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.790vm-test-run-nixbot-gitlab> gitlab # [    4.865247] systemd[1]: Finished Cleanup udev Database.791vm-test-run-nixbot-gitlab> gitlab # [    4.868444] systemd[1]: Reached target Switch Root.792vm-test-run-nixbot-gitlab> gitlab # [    4.871564] systemd[1]: Starting NixOS Activation...793vm-test-run-nixbot-gitlab> gitlab # [    4.970230] initrd-nixos-activation-start[572]: booting system configuration /nix/store/xgnv8xfbgl3ni7xkxqwfk9i51v9ywqly-nixos-system-gitlab-test794vm-test-run-nixbot-gitlab> gitlab # [    5.018760] initrd-nixos-activation-start[572]: running activation script...795vm-test-run-nixbot-gitlab> gitlab # [    5.220862] initrd-nixos-activation-start[595]: setting up /etc...796vm-test-run-nixbot-gitlab> gitlab # [    5.334592] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.797vm-test-run-nixbot-gitlab> gitlab # [    5.338451] systemd[1]: Finished NixOS Activation.798vm-test-run-nixbot-gitlab> gitlab # [    5.347798] systemd[1]: Starting Switch Root...799vm-test-run-nixbot-gitlab> gitlab # [    5.385580] systemd[1]: Switching root.800vm-test-run-nixbot-gitlab> gitlab # [    6.026025] systemd-journald[151]: Received SIGTERM from PID 1 (systemd).801vm-test-run-nixbot-gitlab> gitlab # [    6.185049] NET: Registered PF_VSOCK protocol family802vm-test-run-nixbot-gitlab> gitlab # [    6.561566] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)803vm-test-run-nixbot-gitlab> gitlab # [    6.578002] systemd[1]: Detected virtualization kvm.804vm-test-run-nixbot-gitlab> gitlab # [    6.581226] systemd[1]: Detected architecture x86-64.805vm-test-run-nixbot-gitlab> gitlab # [    6.584581] systemd[1]: Detected first boot.806vm-test-run-nixbot-gitlab> gitlab # [    6.588791] systemd[1]: Initializing machine ID from random generator.807vm-test-run-nixbot-gitlab> gitlab # [    6.729605] systemd[1]: bpf-restrict-fs: LSM BPF program attached808vm-test-run-nixbot-gitlab> gitlab # [    6.798954] systemd[1]: Applying preset policy.809vm-test-run-nixbot-gitlab> gitlab # [    6.966034] systemd[1]: Populated /etc with preset unit settings.810vm-test-run-nixbot-gitlab> gitlab # [    7.082050] systemd[1]: initrd-switch-root.service: Deactivated successfully.811vm-test-run-nixbot-gitlab> gitlab # [    7.084434] systemd[1]: Stopped initrd-switch-root.service.812vm-test-run-nixbot-gitlab> gitlab # [    7.087968] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.813vm-test-run-nixbot-gitlab> gitlab # [    7.091238] systemd[1]: Created slice Slice /system/getty.814vm-test-run-nixbot-gitlab> gitlab # [    7.093394] systemd[1]: Created slice GitLab DevOps Platform Slice.815vm-test-run-nixbot-gitlab> gitlab # [    7.095678] systemd[1]: Created slice User and Session Slice.816vm-test-run-nixbot-gitlab> gitlab # [    7.097220] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.817vm-test-run-nixbot-gitlab> gitlab # [    7.099335] systemd[1]: Started Forward Password Requests to Wall Directory Watch.818vm-test-run-nixbot-gitlab> gitlab # [    7.101235] systemd[1]: Expecting device /dev/hvc0...819vm-test-run-nixbot-gitlab> gitlab # [    7.102551] systemd[1]: Expecting device /dev/ttyS0...820vm-test-run-nixbot-gitlab> gitlab # [    7.103945] systemd[1]: Reached target Local Encrypted Volumes.821vm-test-run-nixbot-gitlab> gitlab # [    7.105401] systemd[1]: Stopped target initrd-fs.target.822vm-test-run-nixbot-gitlab> gitlab # [    7.106872] systemd[1]: Stopped target initrd-root-fs.target.823vm-test-run-nixbot-gitlab> gitlab # [    7.108315] systemd[1]: Stopped target initrd-switch-root.target.824vm-test-run-nixbot-gitlab> gitlab # [    7.109884] systemd[1]: Reached target Virtual Machines and Containers.825vm-test-run-nixbot-gitlab> gitlab # [    7.111541] systemd[1]: Reached target Path Units.826vm-test-run-nixbot-gitlab> gitlab # [    7.112870] systemd[1]: Reached target Remote File Systems.827vm-test-run-nixbot-gitlab> gitlab # [    7.114310] systemd[1]: Reached target Slice Units.828vm-test-run-nixbot-gitlab> gitlab # [    7.115671] systemd[1]: Reached target Swaps.829vm-test-run-nixbot-gitlab> gitlab # [    7.118030] systemd[1]: Listening on Process Core Dump Socket.830vm-test-run-nixbot-gitlab> gitlab # [    7.120159] systemd[1]: Listening on Credential Encryption/Decryption.831vm-test-run-nixbot-gitlab> gitlab # [    7.123512] systemd[1]: Starting Journal Log Access Socket...832vm-test-run-nixbot-gitlab> gitlab # [    7.125793] systemd[1]: Listening on Journal Audit Socket.833vm-test-run-nixbot-gitlab> gitlab # [    7.127440] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.834vm-test-run-nixbot-gitlab> gitlab # [    7.129370] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki835vm-test-run-nixbot-gitlab> gitlab # [    7.131935] systemd[1]: Listening on udev Control Socket.836vm-test-run-nixbot-gitlab> gitlab # [    7.135587] systemd[1]: Mounting Huge Pages File System...837vm-test-run-nixbot-gitlab> gitlab # [    7.138713] systemd[1]: Mounting POSIX Message Queue File System...838vm-test-run-nixbot-gitlab> gitlab # [    7.141917] systemd[1]: Mounting Kernel Debug File System...839vm-test-run-nixbot-gitlab> gitlab # [    7.146036] systemd[1]: Mounting Kernel Trace File System...840vm-test-run-nixbot-gitlab> gitlab # [    7.156201] systemd[1]: Starting Create List of Static Device Nodes...841vm-test-run-nixbot-gitlab> gitlab # [    7.160960] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio842vm-test-run-nixbot-gitlab> gitlab # [    7.174372] systemd[1]: Starting Load Kernel Module configfs...843vm-test-run-nixbot-gitlab> gitlab # [    7.175972] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm844vm-test-run-nixbot-gitlab> gitlab # [    7.178398] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore845vm-test-run-nixbot-gitlab> gitlab # [    7.181065] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse846vm-test-run-nixbot-gitlab> gitlab # [    7.185064] systemd[1]: Mounting FUSE Control File System...847vm-test-run-nixbot-gitlab> gitlab # [    7.198986] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67848vm-test-run-nixbot-gitlab> gitlab # [    7.206106] systemd[1]: Starting Journal Service...849vm-test-run-nixbot-gitlab> gitlab # [    7.210763] systemd[1]: Starting Load Kernel Modules...850vm-test-run-nixbot-gitlab> gitlab # [    7.217968] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...851vm-test-run-nixbot-gitlab> gitlab # [    7.224340] systemd[1]: Starting Remount Root and Kernel File Systems...852vm-test-run-nixbot-gitlab> gitlab # [    7.227831] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki853vm-test-run-nixbot-gitlab> gitlab # [    7.235957] systemd[1]: Starting Coldplug All udev Devices...854vm-test-run-nixbot-gitlab> gitlab # [    7.247228] systemd[1]: Listening on Journal Log Access Socket.855vm-test-run-nixbot-gitlab> gitlab # [    7.249302] systemd[1]: Mounted Huge Pages File System.856vm-test-run-nixbot-gitlab> gitlab # [    7.250963] systemd[1]: Mounted POSIX Message Queue File System.857vm-test-run-nixbot-gitlab> gitlab # [    7.251049] loop: module loaded858vm-test-run-nixbot-gitlab> gitlab # [    7.251577] systemd[1]: Mounted Kernel Debug File System.859vm-test-run-nixbot-gitlab> gitlab # [    7.255006] systemd[1]: Mounted Kernel Trace File System.860vm-test-run-nixbot-gitlab> gitlab # [    7.257395] systemd[1]: Finished Create List of Static Device Nodes.861vm-test-run-nixbot-gitlab> gitlab # [    7.258214] systemd-journald[789]: Collecting audit messages is enabled.862vm-test-run-nixbot-gitlab> gitlab # [    7.259410] systemd[1]: modprobe@configfs.service: Deactivated successfully.863vm-test-run-nixbot-gitlab> gitlab # [    7.263841] systemd[1]: Finished Load Kernel Module configfs.864vm-test-run-nixbot-gitlab> gitlab # [    7.265803] systemd[1]: Mounted FUSE Control File System.865vm-test-run-nixbot-gitlab> gitlab # [    7.267701] EXT4-fs (vda): re-mounted 393e42d7-bbea-4364-93d9-c419e048f38d.866vm-test-run-nixbot-gitlab> gitlab # [    7.268182] systemd[1]: Finished Load Kernel Modules.867vm-test-run-nixbot-gitlab> gitlab # [    7.274935] systemd[1]: Mounting Kernel Configuration File System...868vm-test-run-nixbot-gitlab> gitlab # [    7.013618] systemd[1]: Queued start job for default target Multi-User System[    7.278180] systemd[1]: Starting Firewall...869vm-test-run-nixbot-gitlab> gitlab # .870vm-test-run-nixbot-gitlab> gitlab # [    7.016755] systemd[1]: Unnecessary job was removed for /dev/vdb.871vm-test-run-nixbot-gitlab> gitlab # [    7.018717] systemd[1]: systemd-journald.service: Deactivated successfully.872vm-test-run-nixbot-gitlab> gitlab # [    7.284102] systemd[1]: Starting Apply Kernel Variables...873vm-test-run-nixbot-gitlab> gitlab # [    7.020793] systemd-modules-load[790]: Using 3 probe threads874vm-test-run-nixbot-gitlab> gitlab # [    7.023893] systemd-modules-load[790]: Inserted module 'tls'875vm-test-run-nixbot-gitlab> gitlab # [    7.025917] systemd-modules-load[790]: Insert[    7.289711] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...876vm-test-run-nixbot-gitlab> gitlab # ed module 'loop'877vm-test-run-nixbot-gitlab> gitlab # [    7.029784] systemd-oomd[791]:     7.293493] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.878vm-test-run-nixbot-gitlab> gitlab # ;38:5:185mNo swap; memory pressure usage will be degraded879vm-test-run-nixbot-gitlab> gitlab # [    7.298890] systemd[1]: Started Journal Service.880vm-test-run-nixbot-gitlab> gitlab # [    7.040120] systemd[1]: Finished Remount Root and Kernel File Systems.881vm-test-run-nixbot-gitlab> gitlab # [    7.042757] systemd[1]: Mounted Kernel Configuration File System.882vm-test-run-nixbot-gitlab> gitlab # [    7.048860] systemd[1]: Finished Apply Kernel Variables.883vm-test-run-nixbot-gitlab> gitlab # [    7.052489] systemd[1]: Starting Flush Journal to Persistent Storage...884vm-test-run-nixbot-gitlab> gitlab # [    7.054371] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore885vm-test-run-nixbot-gitlab> gitlab # [    7.059299] systemd[1]: Starting Load/Save OS Random Seed...886vm-test-run-nixbot-gitlab> gitlab # [    7.060774] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki887vm-test-run-nixbot-gitlab> gitlab # [    7.066862] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.888vm-test-run-nixbot-gitlab> gitlab # [    7.337307] systemd-journald[789]: Received client request to flush runtime journal.889vm-test-run-nixbot-gitlab> gitlab # [    7.151331] systemd[1]: Starting Create Static Device Nodes in /dev...890vm-test-run-nixbot-gitlab> gitlab # [    7.154638] systemd[1]: Finished Load/Save OS Random Seed.891vm-test-run-nixbot-gitlab> gitlab # [    7.156430] systemd[1]: Reached target First Boot Complete.892vm-test-run-nixbot-gitlab> gitlab # [    7.158186] systemd[1]: Finished Create Static Device Nodes in /dev.893vm-test-run-nixbot-gitlab> gitlab # [    7.159871] systemd[1]: Reached target Preparation for Local File Systems.894vm-test-run-nixbot-gitlab> gitlab # [    7.161583] systemd[1]: Starting Rule-based Manager for Device Events and Files...895vm-test-run-nixbot-gitlab> gitlab # [    7.163713] systemd-udevd[829]: Using default interface naming scheme 'v260'.896vm-test-run-nixbot-gitlab> gitlab # [    7.165565] systemd[1]: Finished Flush Journal to Persistent Storage.897vm-test-run-nixbot-gitlab> gitlab # [    7.194934] systemd[1]: Finished Coldplug All udev Devices.898vm-test-run-nixbot-gitlab> gitlab # [    7.197681] systemd[1]: Started Rule-based Manager for Device Events and Files.899vm-test-run-nixbot-gitlab> gitlab # [    7.297896] systemd[1]: Finished Firewall.900vm-test-run-nixbot-gitlab> gitlab # [    7.324195] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse901vm-test-run-nixbot-gitlab> gitlab # [    7.352897] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.902vm-test-run-nixbot-gitlab> gitlab # [    7.366555] (udev-worker)[942]: Network interface NamePolicy= disabled on kernel command line.903vm-test-run-nixbot-gitlab> gitlab # [    7.374915] (udev-worker)[919]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.904vm-test-run-nixbot-gitlab> gitlab # [    7.378492] (udev-worker)[919]: Network interface NamePolicy= disabled on kernel command line.905vm-test-run-nixbot-gitlab> gitlab # [    7.382828] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.906vm-test-run-nixbot-gitlab> gitlab # [    7.400757] systemd[1]: Condition check resulted in Virtio network device being skipped.907vm-test-run-nixbot-gitlab> gitlab # [    7.402682] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore908vm-test-run-nixbot-gitlab> gitlab # [    7.405329] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67909vm-test-run-nixbot-gitlab> gitlab # [    7.409196] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore910vm-test-run-nixbot-gitlab> gitlab # [    7.412064] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki911vm-test-run-nixbot-gitlab> gitlab # [    7.414538] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki912vm-test-run-nixbot-gitlab> gitlab # [    7.697128] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3913vm-test-run-nixbot-gitlab> gitlab # [    7.709996] ACPI: button: Power Button [PWRF]914vm-test-run-nixbot-gitlab> gitlab # [    7.738165] rtc_cmos 00:05: RTC can wake from S4915vm-test-run-nixbot-gitlab> gitlab # [    7.739732] mousedev: PS/2 mouse device common for all mice916vm-test-run-nixbot-gitlab> gitlab # [    7.747946] Floppy drive(s): fd0 is 2.88M AMI BIOS917vm-test-run-nixbot-gitlab> gitlab # [    7.750010] rtc_cmos 00:05: registered as rtc0918vm-test-run-nixbot-gitlab> gitlab # [    7.750783] parport_pc 00:03: reported by Plug and Play ACPI919vm-test-run-nixbot-gitlab> gitlab # [    7.751410] rtc_cmos 00:05: setting system clock to 2026-07-04T13:29:42 UTC (1783171782)920vm-test-run-nixbot-gitlab> gitlab # [    7.751896] parport0: PC-style at 0x378921vm-test-run-nixbot-gitlab> gitlab # [    7.752395] systemd-journald[789]: Time jumped backwards, rotating.922vm-test-run-nixbot-gitlab> gitlab # [    7.753108] , irq 7923vm-test-run-nixbot-gitlab> gitlab # [    7.756169] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs924vm-test-run-nixbot-gitlab> gitlab # [    7.761701]  [PCSPP(,...)]925vm-test-run-nixbot-gitlab> gitlab # [    7.763007] FDC 0 is a S82078B926vm-test-run-nixbot-gitlab> gitlab # [    7.765894] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console927vm-test-run-nixbot-gitlab> gitlab # [    7.770963] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0928vm-test-run-nixbot-gitlab> gitlab # [    7.771038] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD929vm-test-run-nixbot-gitlab> gitlab # [    7.785395] Console: switching to colour dummy device 80x25930vm-test-run-nixbot-gitlab> gitlab # [    7.789932] [drm] Found bochs VGA, ID 0xb0c5.931vm-test-run-nixbot-gitlab> gitlab # [    7.790969] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.932vm-test-run-nixbot-gitlab> gitlab # [    7.792860] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic933vm-test-run-nixbot-gitlab> gitlab # [    7.794273] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0934vm-test-run-nixbot-gitlab> gitlab # [    7.800045] fbcon: bochs-drmdrmfb (fb0) is primary device935vm-test-run-nixbot-gitlab> gitlab # [    7.805214] Console: switching to colour frame buffer device 160x50936vm-test-run-nixbot-gitlab> gitlab # [    7.808463] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device937vm-test-run-nixbot-gitlab> gitlab # [    7.817028] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4938vm-test-run-nixbot-gitlab> gitlab # [    7.825008] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6939vm-test-run-nixbot-gitlab> gitlab # [    7.844794] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5940vm-test-run-nixbot-gitlab> gitlab # [    7.609552] systemd[1]: Starting Virtual Console Setup...941vm-test-run-nixbot-gitlab> gitlab # [    7.904951] erofs (device vdb): failed to read inode meta block (nid: 108809984): -4942vm-test-run-nixbot-gitlab> gitlab # [    7.646094] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.943vm-test-run-nixbot-gitlab> gitlab # [    7.648365] systemd[1]: Stopped Virtual Console Setup.944vm-test-run-nixbot-gitlab> gitlab # [    7.661114] systemd[1]: Starting Virtual Console Setup...945vm-test-run-nixbot-gitlab> gitlab # [    7.688126] systemd[1]: syste[    7.952171] ppdev: user-space parallel port driver946vm-test-run-nixbot-gitlab> gitlab # md-vconsole-setup.service: Deactivated successfully.947vm-test-run-nixbot-gitlab> gitlab # [    7.691057] systemd[1]: Stopped Virtual Console Setup.948vm-test-run-nixbot-gitlab> gitlab # [    7.692611] systemd[1]: Starting Virtual Console Setup...949vm-test-run-nixbot-gitlab> gitlab # [    7.982850] kvm_amd: TSC scaling supported950vm-test-run-nixbot-gitlab> gitlab # [    7.983972] kvm_amd: Nested Virtualization enabled951vm-test-run-nixbot-gitlab> gitlab # [    7.985143] kvm_amd: Nested Paging enabled952vm-test-run-nixbot-gitlab> gitlab # [    7.986207] kvm_amd: LBR virtualization supported953vm-test-run-nixbot-gitlab> gitlab # [    7.987468] kvm_amd: Virtual VMLOAD VMSAVE supported954vm-test-run-nixbot-gitlab> gitlab # [    7.988779] kvm_amd: Virtual GIF supported955vm-test-run-nixbot-gitlab> gitlab # [    7.989923] kvm_amd: Virtual NMI enabled956vm-test-run-nixbot-gitlab> gitlab # [    7.774586] systemd-vconsole-setup[962]: Configuration of first virtual console was skipped, ignoring remaining ones.957vm-test-run-nixbot-gitlab> gitlab # [    8.042897] EDAC MC: Ver: 3.0.0958vm-test-run-nixbot-gitlab> gitlab # [    7.780265] systemd[1]: Finished Virtual Console Setup.959vm-test-run-nixbot-gitlab> gitlab # [    7.820617] systemd[1]: Mounting /run/wrappers...960vm-test-run-nixbot-gitlab> gitlab # [    7.837828] systemd[1]: Mounted /run/wrappers.961vm-test-run-nixbot-gitlab> gitlab # [    7.839266] systemd[1]: Reached target Local File Systems.962vm-test-run-nixbot-gitlab> gitlab # [    7.840848] systemd[1]: Listening on Boot Loader Control Service Socket.963vm-test-run-nixbot-gitlab> gitlab # [    7.842647] systemd[1]: Starting register-nix-paths.service...964vm-test-run-nixbot-gitlab> gitlab # [    7.844317] systemd[1]: Starting Create SUID/SGID Wrappers...965vm-test-run-nixbot-gitlab> gitlab # [    7.846046] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.966vm-test-run-nixbot-gitlab> gitlab # [    7.848376] systemd[1]: Starting Save Transient machine-id to Disk...967vm-test-run-nixbot-gitlab> gitlab # [    7.850176] systemd[1]: Starting Create System Files and Directories...968vm-test-run-nixbot-gitlab> gitlab # [    7.880782] systemd[1]: Finished Create System Files and Directories.969vm-test-run-nixbot-gitlab> gitlab # [    7.885410] systemd[1]: Starting Rebuild Journal Catalog...970vm-test-run-nixbot-gitlab> gitlab # [    7.888619] systemd[1]: Starting Record System Boot/Shutdown in UTMP...971vm-test-run-nixbot-gitlab> gitlab # [    7.893152] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.972vm-test-run-nixbot-gitlab> gitlab # [    7.897780] systemd[1]: Finished Save Transient machine-id to Disk.973vm-test-run-nixbot-gitlab> gitlab # [    7.908420] systemd[1]: Finished Record System Boot/Shutdown in UTMP.974vm-test-run-nixbot-gitlab> gitlab # [    7.919877] systemd[1]: Finished Rebuild Journal Catalog.975vm-test-run-nixbot-gitlab> gitlab # [    7.921673] systemd[1]: Starting Update is Completed...976vm-test-run-nixbot-gitlab> gitlab # [    7.937901] systemd[1]: Finished Update is Completed.977vm-test-run-nixbot-gitlab> gitlab # [    7.983862] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.978vm-test-run-nixbot-gitlab> gitlab # [    7.986263] systemd[1]: Finished Create SUID/SGID Wrappers.979vm-test-run-nixbot-gitlab> gitlab # [    7.987662] systemd[1]: suid-sgid-wrappers.service: Consumed 111ms CPU time over 141ms wall clock time, 4.5M memory peak, 1.1M read from disk.980vm-test-run-nixbot-gitlab> gitlab # [    8.257177] systemd[1]: Finished register-nix-paths.service.981vm-test-run-nixbot-gitlab> gitlab # [    8.261841] systemd[1]: Reached target System Initialization.982vm-test-run-nixbot-gitlab> gitlab # [    8.265987] systemd[1]: Started Discard unused filesystem blocks once a week.983vm-test-run-nixbot-gitlab> gitlab # [    8.270543] systemd[1]: Started logrotate.timer.984vm-test-run-nixbot-gitlab> gitlab # [    8.273644] systemd[1]: Started Daily Cleanup of Temporary Directories.985vm-test-run-nixbot-gitlab> gitlab # [    8.277710] systemd[1]: Reached target Timer Units.986vm-test-run-nixbot-gitlab> gitlab # [    8.280866] systemd[1]: Listening on D-Bus System Message Bus Socket.987vm-test-run-nixbot-gitlab> gitlab # [    8.284862] systemd[1]: Listening on Nix Daemon Socket.988vm-test-run-nixbot-gitlab> gitlab # [    8.299970] systemd[1]: Starting nixbot CI service socket...989vm-test-run-nixbot-gitlab> gitlab # [    8.303714] systemd[1]: Listening on Hostname Service Socket.990vm-test-run-nixbot-gitlab> gitlab # [    8.308395] systemd[1]: Starting D-Bus System Message Bus...991vm-test-run-nixbot-gitlab> gitlab # [    8.312205] systemd[1]: Listening on nixbot CI service socket.992vm-test-run-nixbot-gitlab> gitlab # [    8.315715] systemd[1]: Reached target Socket Units.993vm-test-run-nixbot-gitlab> gitlab # [    8.364554] dbus-broker-launch[1052]: Looking up NSS user entry for 'systemd-timesync'...994vm-test-run-nixbot-gitlab> gitlab # [    8.370359] dbus-broker-launch[1052]: NSS returned no entry for 'systemd-timesync'995vm-test-run-nixbot-gitlab> gitlab # [    8.374834] dbus-broker-launch[1052]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"996vm-test-run-nixbot-gitlab> gitlab # [    8.384601] systemd[1]: Started D-Bus System Message Bus.997vm-test-run-nixbot-gitlab> gitlab # [    8.390565] dbus-broker-launch[1052]: Ready998vm-test-run-nixbot-gitlab> gitlab # [    8.393640] systemd[1]: Reached target Basic System.999vm-test-run-nixbot-gitlab> gitlab # [    8.397148] systemd[1]: Started backdoor.service.1000vm-test-run-nixbot-gitlab> gitlab # [    8.400760] systemd[1]: Starting gitlab-config.service...1001vm-test-run-nixbot-gitlab> gitlab # [    8.404655] systemd[1]: Starting Import lastlog data into lastlog2 database...1002vm-test-run-nixbot-gitlab> gitlab # [    8.408689] systemd[1]: Starting Logrotate configuration check...1003vm-test-run-nixbot-gitlab> gitlab # [    8.411959] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1004vm-test-run-nixbot-gitlab> gitlab # [    8.415449] systemd[1]: Starting Post-Boot Actions...1005vm-test-run-nixbot-gitlab> gitlab # [    8.424406] systemd[1]: Started Reset console on configuration changes.1006vm-test-run-nixbot-gitlab> gitlab # [    8.428927] systemd[1]: Starting resolvconf update...1007vm-test-run-nixbot-gitlab> gitlab # [    8.432462] logrotate[1057]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.1008vm-test-run-nixbot-gitlab> gitlab # [    8.436218] logrotate[1057]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf1009vm-test-run-nixbot-gitlab> gitlab # [    8.438558] logrotate[1057]: note: 'monthly' overrides previously specified 'weekly'1010vm-test-run-nixbot-gitlab> gitlab # [    8.440553] logrotate[1057]: Reading state from file: /var/lib/logrotate.status1011vm-test-run-nixbot-gitlab> gitlab # [    8.442319] logrotate[1057]: state file /var/lib/logrotate.status does not exist1012vm-test-run-nixbot-gitlab> gitlab # [    8.444183] logrotate[1057]: Allocating hash table for state file, size 64 entries1013vm-test-run-nixbot-gitlab> gitlab # [    8.446185] logrotate[1057]: Handling 4 logs1014vm-test-run-nixbot-gitlab> gitlab # [    8.447381] logrotate[1057]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1015vm-test-run-nixbot-gitlab> gitlab # [    8.450978] logrotate[1057]: considering log /var/log/btmp1016vm-test-run-nixbot-gitlab> gitlab # [    8.453041] logrotate[1057]: Creating new state1017vm-test-run-nixbot-gitlab> gitlab # [    8.454168] logrotate[1057]:   Now: 2026-07-04 13:291018vm-test-run-nixbot-gitlab> gitlab # [    8.455504] logrotate[1057]:   Last rotated at 2026-07-04 13:001019vm-test-run-nixbot-gitlab> gitlab # [    8.456878] logrotate[1057]:   log does not need rotating (log has already been rotated)1020vm-test-run-nixbot-gitlab> gitlab # [    8.460236] logrotate[1057]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed1021vm-test-run-nixbot-gitlab> gitlab # connecting to host...1022vm-test-run-nixbot-gitlab> gitlab # [    8.464560] logrotate[1057]: considering log /var/log/wtmp1023vm-test-run-nixbot-gitlab> gitlab # [    8.466114] logrotate[1057]: Creating new state1024vm-test-run-nixbot-gitlab> gitlab # [    8.467474] logrotate[1057]:   Now: 2026-07-04 13:291025vm-test-run-nixbot-gitlab> gitlab # [    8.468239] logrotate[1057]:   Last rotated at 2026-07-04 13:001026vm-test-run-nixbot-gitlab> gitlab # [    8.469129] logrotate[1057]:   log does not need rotating (log has already been rotated)1027vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'1028vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell1029vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.33 seconds)1030vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.33 seconds)1031vm-test-run-nixbot-gitlab> gitlab # [    8.469982] logrotate[1057]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed1032vm-test-run-nixbot-gitlab> gitlab # [    8.471318] logrotate[1057]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1057)1033vm-test-run-nixbot-gitlab> gitlab # [    8.472097] logrotate[1057]: considering log /var/gitlab/state/log/*.log1034vm-test-run-nixbot-gitlab> gitlab # [    8.472926] logrotate[1057]:   log /var/gitlab/state/log/*.log does not exist -- skipping1035vm-test-run-nixbot-gitlab> gitlab # [    8.474132] logrotate[1057]: Creating new state1036vm-test-run-nixbot-gitlab> gitlab # [    8.474942] logrotate[1057]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1057)1037vm-test-run-nixbot-gitlab> gitlab # [    8.476512] nsncd[1058]: Jul 04 13:29:43.458 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1038vm-test-run-nixbot-gitlab> gitlab # [    8.477587] systemd[1]: Finished Logrotate configuration check.1039vm-test-run-nixbot-gitlab> gitlab # [    8.478865] logrotate[1057]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed1040vm-test-run-nixbot-gitlab> gitlab # [    8.480147] logrotate[1057]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1057)1041vm-test-run-nixbot-gitlab> gitlab # [    8.480864] logrotate[1057]: considering log /var/log/nginx/*.log1042vm-test-run-nixbot-gitlab> gitlab # [    8.481962] logrotate[1057]:   log /var/log/nginx/*.log does not exist -- skipping1043vm-test-run-nixbot-gitlab> gitlab # [    8.483104] logrotate[1057]: Creating new state1044vm-test-run-nixbot-gitlab> gitlab # [    8.483849] logrotate[1057]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1057)1045vm-test-run-nixbot-gitlab> gitlab # [    8.485150] systemd[1]: Finished Post-Boot Actions.1046vm-test-run-nixbot-gitlab> gitlab # [    8.486652] systemd[1]: Started Name Service Cache Daemon (nsncd).1047vm-test-run-nixbot-gitlab> gitlab # [    8.488542] systemd[1]: Reached target Host and Network Name Lookups.1048vm-test-run-nixbot-gitlab> gitlab # [    8.489787] systemd[1]: Reached target User and Group Name Lookups.1049vm-test-run-nixbot-gitlab> gitlab # [    8.491487] systemd[1]: Starting User Login Management...1050vm-test-run-nixbot-gitlab> gitlab # [    8.508083] systemd[1]: Finished Import lastlog data into lastlog2 database.1051vm-test-run-nixbot-gitlab> gitlab # [    8.536964] systemd[1]: Stopped target Host and Network Name Lookups.1052vm-test-run-nixbot-gitlab> gitlab # [    8.537866] systemd[1]: Stopping Host and Network Name Lookups...1053vm-test-run-nixbot-gitlab> gitlab # [    8.539055] systemd[1]: Stopped target User and Group Name Lookups.1054vm-test-run-nixbot-gitlab> gitlab # [    8.539919] systemd[1]: Stopping User and Group Name Lookups...1055vm-test-run-nixbot-gitlab> gitlab # [    8.541253] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...1056vm-test-run-nixbot-gitlab> gitlab # [    8.545144] systemd[1]: nscd.service: Deactivated successfully.1057vm-test-run-nixbot-gitlab> gitlab # [    8.545964] systemd[1]: Stopped Name Service Cache Daemon (nsncd).1058vm-test-run-nixbot-gitlab> gitlab # [    8.547164] systemd[1]: nscd.service: Consumed 24ms CPU time over 138ms wall clock time, 7.8M memory peak, 5.6M read from disk.1059vm-test-run-nixbot-gitlab> gitlab # [    8.553611] systemd[1]: Starting Name Service Cache Daemon (nsncd)...1060vm-test-run-nixbot-gitlab> gitlab # [    8.558798] systemd-logind[1084]: New seat seat0.1061vm-test-run-nixbot-gitlab> gitlab # [    8.571927] systemd-logind[1084]: Watching system buttons on /dev/input/event2 (Power Button)1062vm-test-run-nixbot-gitlab> gitlab # [    8.573402] systemd-logind[1084]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)1063vm-test-run-nixbot-gitlab> gitlab # [    8.574282] systemd-logind[1084]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)1064vm-test-run-nixbot-gitlab> gitlab # [    8.575318] systemd[1]: Started User Login Management.1065vm-test-run-nixbot-gitlab> gitlab # [    8.576830] nsncd[1141]: Jul 04 13:29:43.588 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"1066vm-test-run-nixbot-gitlab> gitlab # [    8.578174] systemd[1]: Starting linger-users.service...1067vm-test-run-nixbot-gitlab> gitlab # [    8.579715] systemd[1]: Started Name Service Cache Daemon (nsncd).1068vm-test-run-nixbot-gitlab> gitlab # [    8.580794] systemd[1]: Reached target Host and Network Name Lookups.1069vm-test-run-nixbot-gitlab> gitlab # [    8.581894] systemd[1]: Reached target User and Group Name Lookups.1070vm-test-run-nixbot-gitlab> gitlab # [    8.599669] systemd[1]: Finished resolvconf update.1071vm-test-run-nixbot-gitlab> gitlab # [    8.600809] systemd[1]: linger-users.service: Deactivated successfully.1072vm-test-run-nixbot-gitlab> gitlab # [    8.601561] systemd[1]: Finished linger-users.service.1073vm-test-run-nixbot-gitlab> gitlab # [    8.604342] systemd[1]: Reached target Preparation for Network.1074vm-test-run-nixbot-gitlab> gitlab # [    8.608159] systemd[1]: Starting DHCP Client...1075vm-test-run-nixbot-gitlab> gitlab # [    8.609585] systemd[1]: Starting Address configuration of eth1...1076vm-test-run-nixbot-gitlab> gitlab # [    8.613470] systemd[1]: Starting Extra networking commands....1077vm-test-run-nixbot-gitlab> gitlab # [    8.637452] network-addresses-eth1-start[1170]: adding address 192.168.1.1/24... done1078vm-test-run-nixbot-gitlab> gitlab # [    8.643366] network-addresses-eth1-start[1170]: adding address 2001:db8:1::1/64... done1079vm-test-run-nixbot-gitlab> gitlab # [    8.649704] systemd[1]: Finished Address configuration of eth1.1080vm-test-run-nixbot-gitlab> gitlab # [    8.680585] systemd[1]: Finished Extra networking commands..1081vm-test-run-nixbot-gitlab> gitlab # [    8.681469] systemd[1]: Reached target Network.1082vm-test-run-nixbot-gitlab> gitlab # [    8.683666] systemd[1]: Starting gitlab-workhorse.service...1083vm-test-run-nixbot-gitlab> gitlab # [    8.686347] dhcpcd[1178]: dhcpcd-10.3.2 starting1084vm-test-run-nixbot-gitlab> gitlab # [    8.687342] systemd[1]: Starting Nginx Web Server...1085vm-test-run-nixbot-gitlab> gitlab # [    8.690555] systemd[1]: Starting PostgreSQL Server...1086vm-test-run-nixbot-gitlab> gitlab # [    8.693525] dhcpcd[1230]: dev: loaded udev1087vm-test-run-nixbot-gitlab> gitlab # [    8.960769] 8021q: 802.1Q VLAN Support v1.81088vm-test-run-nixbot-gitlab> gitlab # [    8.961582] 8021q: adding VLAN 0 to HW filter on device eth11089vm-test-run-nixbot-gitlab> gitlab # [    8.700147] systemd[1]: Starting Redis Server - redis-gitlab...1090vm-test-run-nixbot-gitlab> gitlab # [    8.704410] systemd[1]: Starting Permit User Sessions...1091vm-test-run-nixbot-gitlab> gitlab # [    8.726120] systemd[1]: Finished Permit User Sessions.1092vm-test-run-nixbot-gitlab> gitlab # [    8.730246] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.1093vm-test-run-nixbot-gitlab> gitlab # [    8.739282] systemd[1]: Started Getty on tty1.1094vm-test-run-nixbot-gitlab> gitlab # [    8.740250] systemd[1]: Reached target Login Prompts.1095vm-test-run-nixbot-gitlab> gitlab # [    9.048455] cfg80211: Loading compiled-in X.509 certificates for regulatory database1096vm-test-run-nixbot-gitlab> gitlab # [    9.055997] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'1097vm-test-run-nixbot-gitlab> gitlab # [    9.057788] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'1098vm-test-run-nixbot-gitlab> gitlab # [    9.061470] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -21099vm-test-run-nixbot-gitlab> gitlab # [    9.063585] cfg80211: failed to load regulatory.db1100vm-test-run-nixbot-gitlab> gitlab # [    9.098088] 8021q: adding VLAN 0 to HW filter on device eth01101vm-test-run-nixbot-gitlab> gitlab # [    8.836579] dhcpcd[1230]: eth0: waiting for carrier1102vm-test-run-nixbot-gitlab> gitlab # [    8.837354] dhcpcd[1230]: eth0: carrier acquired1103vm-test-run-nixbot-gitlab> gitlab # [    8.842540] dhcpcd[1230]: DUID 00:01:00:01:31:db:c7:47:52:54:00:12:34:561104vm-test-run-nixbot-gitlab> gitlab # [    8.842820] dhcpcd[1230]: eth0: IAID 00:12:34:561105vm-test-run-nixbot-gitlab> gitlab # [    8.844226] dhcpcd[1230]: eth0: adding address fe80::5054:ff:fe12:34561106vm-test-run-nixbot-gitlab> gitlab # [    8.854352] nginx-pre-start[1254]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok1107vm-test-run-nixbot-gitlab> gitlab # [    8.856509] nginx-pre-start[1254]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful1108vm-test-run-nixbot-gitlab> gitlab # [    8.859133] postgresql-pre-start[1259]: The files belonging to this database system will be owned by user "postgres".1109vm-test-run-nixbot-gitlab> gitlab # [    8.860090] postgresql-pre-start[1259]: This user must also own the server process.1110vm-test-run-nixbot-gitlab> gitlab # [    8.861988] postgresql-pre-start[1259]: The database cluster will be initialized with locale "en_US.UTF-8".1111vm-test-run-nixbot-gitlab> gitlab # [    8.862844] postgresql-pre-start[1259]: The default database encoding has accordingly been set to "UTF8".1112vm-test-run-nixbot-gitlab> gitlab # [    8.864132] postgresql-pre-start[1259]: The default text search configuration will be set to "english".1113vm-test-run-nixbot-gitlab> gitlab # [    8.865069] postgresql-pre-start[1259]: Data page checksums are disabled.1114vm-test-run-nixbot-gitlab> gitlab # [    8.865764] postgresql-pre-start[1259]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1115vm-test-run-nixbot-gitlab> gitlab # [    8.867545] postgresql-pre-start[1259]: creating subdirectories ... ok1116vm-test-run-nixbot-gitlab> gitlab # [    8.868409] postgresql-pre-start[1259]: selecting dynamic shared memory implementation ... posix1117vm-test-run-nixbot-gitlab> gitlab # [    8.883530] redis-server[1252]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1118vm-test-run-nixbot-gitlab> gitlab # [    8.887085] redis-server[1252]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1119vm-test-run-nixbot-gitlab> gitlab # [    8.888223] redis-server[1252]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1252, just started1120vm-test-run-nixbot-gitlab> gitlab # [    8.889174] redis-server[1252]: Configuration loaded1121vm-test-run-nixbot-gitlab> gitlab # [    8.889862] redis[1252]: monotonic clock: POSIX clock_gettime1122vm-test-run-nixbot-gitlab> gitlab # [    8.891306] redis[1252]: Running mode=standalone, port=0.1123vm-test-run-nixbot-gitlab> gitlab # [    8.892514] redis[1252]: Server initialized1124vm-test-run-nixbot-gitlab> gitlab # [    8.893213] redis[1252]: Ready to accept connections unix1125vm-test-run-nixbot-gitlab> gitlab # [    8.894133] redis[1252]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1126vm-test-run-nixbot-gitlab> gitlab # [    8.903219] postgresql-pre-start[1259]: selecting default "max_connections" ... 1001127vm-test-run-nixbot-gitlab> gitlab # [    8.904362] systemd[1]: Started Nginx Web Server.1128vm-test-run-nixbot-gitlab> gitlab # [    8.909867] systemd[1]: Started Redis Server - redis-gitlab.1129vm-test-run-nixbot-gitlab> gitlab # [    8.938097] postgresql-pre-start[1259]: selecting default "shared_buffers" ... 128MB1130vm-test-run-nixbot-gitlab> gitlab # [    9.173789] systemd[1]: Started gitlab-workhorse.service.1131vm-test-run-nixbot-gitlab> gitlab # [    9.344571] postgresql-pre-start[1259]: selecting default time zone ... UTC1132vm-test-run-nixbot-gitlab> gitlab # [    9.346502] postgresql-pre-start[1259]: creating configuration files ... ok1133vm-test-run-nixbot-gitlab> gitlab # [    9.440218] gitlab-workhorse[1288]: time="2026-07-04T13:29:44Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41134vm-test-run-nixbot-gitlab> gitlab # [    9.441439] gitlab-workhorse[1288]: time="2026-07-04T13:29:44Z" level=info msg="Using redis/go-redis"1135vm-test-run-nixbot-gitlab> gitlab # [    9.445256] gitlab-workhorse[1288]: time="2026-07-04T13:29:44Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1136vm-test-run-nixbot-gitlab> gitlab # [    9.518267] postgresql-pre-start[1259]: running bootstrap script ... ok1137vm-test-run-nixbot-gitlab> gitlab # [    9.957119] postgresql-pre-start[1259]: performing post-bootstrap initialization ... ok1138vm-test-run-nixbot-gitlab> gitlab # [   10.219405] dhcpcd[1230]: eth0: soliciting a DHCP lease1139vm-test-run-nixbot-gitlab> gitlab # [   10.499165] NET: Registered PF_PACKET protocol family1140vm-test-run-nixbot-gitlab> gitlab # [   10.245497] dhcpcd[1230]: eth0: offered 10.0.2.15 from 10.0.2.21141vm-test-run-nixbot-gitlab> gitlab # [   10.252547] dhcpcd[1230]: eth0: probing address 10.0.2.15/241142vm-test-run-nixbot-gitlab> gitlab # [   10.322923] postgresql-pre-start[1259]: syncing data to disk ... ok1143vm-test-run-nixbot-gitlab> gitlab # [   10.324753] postgresql-pre-start[1259]: initdb: warning: enabling "trust" authentication for local connections1144vm-test-run-nixbot-gitlab> gitlab # [   10.327229] postgresql-pre-start[1259]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1145vm-test-run-nixbot-gitlab> gitlab # [   10.331047] postgresql-pre-start[1259]: Success. You can now start the database server using:1146vm-test-run-nixbot-gitlab> gitlab # [   10.333206] postgresql-pre-start[1259]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1147vm-test-run-nixbot-gitlab> gitlab # [   10.412409] postgres[1315]: [1315] LOG:  starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1148vm-test-run-nixbot-gitlab> gitlab # [   10.420186] postgres[1315]: [1315] LOG:  listening on IPv6 address "::1", port 54321149vm-test-run-nixbot-gitlab> gitlab # [   10.425412] postgres[1315]: [1315] LOG:  listening on IPv4 address "127.0.0.1", port 54321150vm-test-run-nixbot-gitlab> gitlab # [   10.430645] postgres[1315]: [1315] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1151vm-test-run-nixbot-gitlab> gitlab # [   10.436948] postgres[1321]: [1321] LOG:  database system was shut down at 2026-07-04 13:29:44 GMT1152vm-test-run-nixbot-gitlab> gitlab # [   10.442432] postgres[1315]: [1315] LOG:  database system is ready to accept connections1153vm-test-run-nixbot-gitlab> gitlab # [   10.447196] systemd[1]: Started PostgreSQL Server.1154vm-test-run-nixbot-gitlab> gitlab # [   10.450926] systemd[1]: Starting PostgreSQL Setup Scripts...1155vm-test-run-nixbot-gitlab> gitlab # [   10.642617] postgresql-setup-start[1332]: CREATE DATABASE1156vm-test-run-nixbot-gitlab> gitlab # [   10.669706] postgresql-setup-start[1346]: CREATE ROLE1157vm-test-run-nixbot-gitlab> gitlab # [   10.682583] postgresql-setup-start[1348]: ALTER DATABASE1158vm-test-run-nixbot-gitlab> gitlab # [   10.703769] postgresql-setup-start[1353]: CREATE ROLE1159vm-test-run-nixbot-gitlab> gitlab # [   10.708519] systemd[1]: Finished PostgreSQL Setup Scripts.1160vm-test-run-nixbot-gitlab> gitlab # [   10.710086] systemd[1]: Reached target PostgreSQL.1161vm-test-run-nixbot-gitlab> gitlab # [   10.712389] systemd[1]: Starting gitlab-postgresql.service...1162vm-test-run-nixbot-gitlab> gitlab # [   10.817874] gitlab-postgresql-start[1361]: CREATE DATABASE1163vm-test-run-nixbot-gitlab> gitlab # [   10.856986] gitlab-postgresql-start[1365]: CREATE EXTENSION1164vm-test-run-nixbot-gitlab> gitlab # [   10.897221] gitlab-postgresql-start[1371]: CREATE EXTENSION1165vm-test-run-nixbot-gitlab> gitlab # [   10.901382] systemd[1]: Finished gitlab-postgresql.service.1166vm-test-run-nixbot-gitlab> gitlab # [   10.943479] dhcpcd[1230]: eth0: soliciting an IPv6 router1167vm-test-run-nixbot-gitlab> gitlab # [   10.947542] dhcpcd[1230]: eth0: Router Advertisement from fe80::21168vm-test-run-nixbot-gitlab> gitlab # [   10.951576] dhcpcd[1230]: eth0: adding address fec0::5054:ff:fe12:3456/641169vm-test-run-nixbot-gitlab> gitlab # [   10.955925] dhcpcd[1230]: eth0: adding route to fec0::/641170vm-test-run-nixbot-gitlab> gitlab # [   10.959890] dhcpcd[1230]: eth0: adding default route via fe80::21171vm-test-run-nixbot-gitlab> gitlab # [   10.998494] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1369]: mkdir -p /var/gitlab/state/home/.ssh: OK1172vm-test-run-nixbot-gitlab> gitlab # [   11.000845] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1369]: chmod 700 /var/gitlab/state/home/.ssh: OK1173vm-test-run-nixbot-gitlab> gitlab # [   11.075709] systemd[1]: Finished gitlab-config.service.1174vm-test-run-nixbot-gitlab> gitlab # [   11.082219] systemd[1]: Started gitaly.service.1175vm-test-run-nixbot-gitlab> gitlab # [   11.085891] systemd[1]: Starting gitlab-db-config.service...1176vm-test-run-nixbot-gitlab> gitlab # [   11.634882] gitaly[1387]: time="2026-07-04T13:29:46.646Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=13871177vm-test-run-nixbot-gitlab> gitlab # [   11.638093] gitaly[1387]: time="2026-07-04T13:29:46.647Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13871178vm-test-run-nixbot-gitlab> gitlab # [   11.641207] gitaly[1387]: time="2026-07-04T13:29:46.647Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13871179vm-test-run-nixbot-gitlab> gitlab # [   11.644117] gitaly[1387]: time="2026-07-04T13:29:46.647Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13871180vm-test-run-nixbot-gitlab> gitlab # [   11.647081] gitaly[1387]: time="2026-07-04T13:29:46.647Z" level=info msg="Starting Gitaly" pid=1387 version=18.11.41181vm-test-run-nixbot-gitlab> gitlab # [   11.815520] gitaly[1387]: time="2026-07-04T13:29:46.826Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=178 pid=13871182vm-test-run-nixbot-gitlab> gitlab # [   11.816206] gitaly[1387]: time="2026-07-04T13:29:46.827Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13871183vm-test-run-nixbot-gitlab> gitlab # [   11.817988] gitaly[1387]: time="2026-07-04T13:29:46.827Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13871184vm-test-run-nixbot-gitlab> gitlab # [   11.818853] gitaly[1387]: time="2026-07-04T13:29:46.827Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1387/git-exec-3485459523.d/git pid=13871185vm-test-run-nixbot-gitlab> gitlab # [   11.827138] gitaly[1387]: time="2026-07-04T13:29:46.839Z" level=info msg="finished detecting git version" duration_ms=11 pid=13871186vm-test-run-nixbot-gitlab> gitlab # [   11.828458] gitaly[1387]: time="2026-07-04T13:29:46.840Z" level=info msg="using Git version" pid=1387 version=2.53.g1187vm-test-run-nixbot-gitlab> gitlab # [   11.830607] gitaly[1387]: time="2026-07-04T13:29:46.842Z" level=info msg="clearing disk cache object folder" pid=1387 storage=default1188vm-test-run-nixbot-gitlab> gitlab # [   11.831322] gitaly[1387]: time="2026-07-04T13:29:46.842Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1309030330 pid=1387 storage=default1189vm-test-run-nixbot-gitlab> gitlab # [   11.832188] gitaly[1387]: time="2026-07-04T13:29:46.842Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1309030330 pid=1387 storage=default1190vm-test-run-nixbot-gitlab> gitlab # [   11.833243] gitaly[1387]: time="2026-07-04T13:29:46.843Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13871191vm-test-run-nixbot-gitlab> gitlab # [   11.834201] gitaly[1387]: time="2026-07-04T13:29:46.844Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13871192vm-test-run-nixbot-gitlab> gitlab # [   11.835084] gitaly[1387]: time="2026-07-04T13:29:46.844Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1309030330 clear_duration_ms=0 pid=1387 storage=default1193vm-test-run-nixbot-gitlab> gitlab # [   11.911675] gitaly[1387]: time="2026-07-04T13:29:46.923Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13871194vm-test-run-nixbot-gitlab> gitlab # [   11.913471] gitaly[1387]: time="2026-07-04T13:29:46.923Z" level=info msg="finished tempdir cleaner walk" pid=1387 storage=default time_ms=01195vm-test-run-nixbot-gitlab> gitlab # [   11.915240] gitaly[1387]: time="2026-07-04T13:29:46.923Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1387 schema=unix1196vm-test-run-nixbot-gitlab> gitlab # [   11.917527] gitaly[1387]: time="2026-07-04T13:29:46.923Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1387/sock.d/intern pid=1387 schema=unix1197vm-test-run-nixbot-gitlab> gitlab # [   11.919583] gitaly[1387]: time="2026-07-04T13:29:46.923Z" level=info msg="counting repositories" pid=13871198vm-test-run-nixbot-gitlab> gitlab # [   11.921397] gitaly[1387]: time="2026-07-04T13:29:46.924Z" level=info msg="starting to count repositories" pid=1387 storage_path=/var/gitlab/state/repositories1199vm-test-run-nixbot-gitlab> gitlab # [   11.930862] gitaly[1387]: time="2026-07-04T13:29:46.942Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1387 storage_path=/var/gitlab/state/repositories1200vm-test-run-nixbot-gitlab> gitlab # [   11.931538] gitaly[1387]: time="2026-07-04T13:29:46.942Z" level=info msg="completed counting all repositories" counting_duration=18 pid=13871201vm-test-run-nixbot-gitlab> gitlab # [   11.932499] gitaly[1387]: time="2026-07-04T13:29:46.943Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13871202vm-test-run-nixbot-gitlab> gitlab # [   11.933960] gitaly[1387]: time="2026-07-04T13:29:46.943Z" level=info msg="Started Gitaly" duration_ms=295 pid=13871203vm-test-run-nixbot-gitlab> gitlab # [   11.935842] gitaly[1387]: time="2026-07-04T13:29:46.943Z" level=info msg="maintenance: daily scheduled" pid=1387 scheduled="2026-07-05 12:00:00 +0000 UTC"1204vm-test-run-nixbot-gitlab> gitlab # [   12.584488] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1205vm-test-run-nixbot-gitlab> gitlab # [   12.585668] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1206vm-test-run-nixbot-gitlab> gitlab # [   12.612729] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1207vm-test-run-nixbot-gitlab> gitlab # [   13.898509] gitaly[1387]: time="2026-07-04T13:29:48.909Z" level=info msg="License database preloaded" duration_ms=2206 pid=13871208vm-test-run-nixbot-gitlab> gitlab # [   15.524421] dhcpcd[1230]: eth0: leased 10.0.2.15 for 86400 seconds1209vm-test-run-nixbot-gitlab> gitlab # [   15.525785] dhcpcd[1230]: eth0: adding route to 10.0.2.0/241210vm-test-run-nixbot-gitlab> gitlab # [   15.528515] dhcpcd[1230]: eth0: adding default route via 10.0.2.21211vm-test-run-nixbot-gitlab> gitlab # [   15.634350] systemd[1]: Started DHCP Client.1212vm-test-run-nixbot-gitlab> gitlab # [   15.637723] systemd[1]: Reached target Network is Online.1213vm-test-run-nixbot-gitlab> gitlab # [   44.444118] postgres[1754]: [1754] ERROR:  relation "feature_gates" does not exist at character 5231214vm-test-run-nixbot-gitlab> gitlab # [   44.444570] postgres[1754]: [1754] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1215vm-test-run-nixbot-gitlab> gitlab # [   44.453327] postgres[1754]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1216vm-test-run-nixbot-gitlab> gitlab # [   44.462378] postgres[1754]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1217vm-test-run-nixbot-gitlab> gitlab # [   44.474350] postgres[1754]: 	       attidentity AS identity,1218vm-test-run-nixbot-gitlab> gitlab # [   44.481209] postgres[1754]: 	       attgenerated as attgenerated1219vm-test-run-nixbot-gitlab> gitlab # [   44.485355] postgres[1754]: 	  FROM pg_attribute a1220vm-test-run-nixbot-gitlab> gitlab # [   44.486863] postgres[1754]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1221vm-test-run-nixbot-gitlab> gitlab # [   44.491900] postgres[1754]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1222vm-test-run-nixbot-gitlab> gitlab # [   44.494128] postgres[1754]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1223vm-test-run-nixbot-gitlab> gitlab # [   44.497231] postgres[1754]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1224vm-test-run-nixbot-gitlab> gitlab # [   44.504213] postgres[1754]: 	   AND a.attnum > 0 AND NOT a.attisdropped1225vm-test-run-nixbot-gitlab> gitlab # [   44.510089] postgres[1754]: 	 ORDER BY a.attnum1226vm-test-run-nixbot-gitlab> gitlab # [   44.514250] postgres[1754]:1227vm-test-run-nixbot-gitlab> gitlab # [   44.669296] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1390]: W, [2026-07-04T13:30:19.680078 #1390]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1228vm-test-run-nixbot-gitlab> gitlab # [   49.518175] gitlab-workhorse[1288]: time="2026-07-04T13:30:24Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1229vm-test-run-nixbot-gitlab> gitlab # [   50.422344] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1818]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1230vm-test-run-nixbot-gitlab> gitlab # [   50.423233] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1818]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1231vm-test-run-nixbot-gitlab> gitlab # [   59.527454] gitlab-workhorse[1288]: time="2026-07-04T13:30:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1232vm-test-run-nixbot-gitlab> gitlab # [   67.672974] postgres[1893]: [1893] ERROR:  relation "table_without_models" does not exist1233vm-test-run-nixbot-gitlab> gitlab # [   67.673644] postgres[1893]: [1893] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1234vm-test-run-nixbot-gitlab> gitlab # [   69.538934] gitlab-workhorse[1288]: time="2026-07-04T13:30:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1235vm-test-run-nixbot-gitlab> gitlab # [   79.548357] gitlab-workhorse[1288]: time="2026-07-04T13:30:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1236vm-test-run-nixbot-gitlab> gitlab # [   83.505171] systemd[1]: Finished gitlab-db-config.service.1237vm-test-run-nixbot-gitlab> gitlab # [   83.514282] systemd[1]: Started gitlab-sidekiq.service.1238vm-test-run-nixbot-gitlab> gitlab # [   83.517227] systemd[1]: Starting gitlab.service...1239vm-test-run-nixbot-gitlab> gitlab # [   83.852725] bundle[2107]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1240vm-test-run-nixbot-gitlab> gitlab # [   83.854525] bundle[2107]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1241vm-test-run-nixbot-gitlab> gitlab # [   83.860136] bundle[2107]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1242vm-test-run-nixbot-gitlab> gitlab # [   86.762891] sidekiq-cluster[2104]: {"severity":"INFO","time":"2026-07-04T13:31:01.756Z","message":"Starting cluster with 1 processes"}1243vm-test-run-nixbot-gitlab> gitlab # [   86.878200] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.868Z","pid":2107,"message":"* Enabling systemd notification integration"}1244vm-test-run-nixbot-gitlab> gitlab # [   86.879404] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"Puma starting in cluster mode..."}1245vm-test-run-nixbot-gitlab> gitlab # [   86.881139] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1246vm-test-run-nixbot-gitlab> gitlab # [   86.883104] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1247vm-test-run-nixbot-gitlab> gitlab # [   86.884524] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*  Min threads: 0"}1248vm-test-run-nixbot-gitlab> gitlab # [   86.885890] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*  Max threads: 4"}1249vm-test-run-nixbot-gitlab> gitlab # [   86.887438] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*  Environment: production"}1250vm-test-run-nixbot-gitlab> gitlab # [   86.889305] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*   Master PID: 2107"}1251vm-test-run-nixbot-gitlab> gitlab # [   86.890888] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*      Workers: 2"}1252vm-test-run-nixbot-gitlab> gitlab # [   86.892418] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1253vm-test-run-nixbot-gitlab> gitlab # [   86.893846] bundle[2107]: {"timestamp":"2026-07-04T13:31:01.889Z","pid":2107,"message":"* Preloading application"}1254vm-test-run-nixbot-gitlab> gitlab # [   89.550741] gitlab-workhorse[1288]: time="2026-07-04T13:31:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1255vm-test-run-nixbot-gitlab> gitlab # [   99.552604] gitlab-workhorse[1288]: time="2026-07-04T13:31:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1256vm-test-run-nixbot-gitlab> gitlab # [  109.562753] gitlab-workhorse[1288]: time="2026-07-04T13:31:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1257vm-test-run-nixbot-gitlab> gitlab # [  110.895878] bundle[2107]: W, [2026-07-04T13:31:25.906905 #2107]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1258vm-test-run-nixbot-gitlab> gitlab # [  113.459630] sidekiq-cluster[2137]: 2026-07-04T13:31:28.470Z pid=2137 tid=tzl INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2137, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425541632}1259vm-test-run-nixbot-gitlab> gitlab # [  113.643946] sidekiq-cluster[2137]: W, [2026-07-04T13:31:28.655611 #2137]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1260vm-test-run-nixbot-gitlab> gitlab # [  113.684539] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:28.696Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1261vm-test-run-nixbot-gitlab> gitlab # [  115.537496] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.548Z","message":"GitLab reliable fetch activated!"}1262vm-test-run-nixbot-gitlab> gitlab # [  115.538650] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.548Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1263vm-test-run-nixbot-gitlab> gitlab # [  115.565756] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.577Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  115.569673] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.581Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  115.573684] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.585Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  115.577671] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.589Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  115.581690] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.593Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  115.585511] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.597Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  115.588976] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.600Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  115.592696] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.604Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  115.596299] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.608Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  115.599620] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.611Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  115.603175] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.615Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  115.607353] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.619Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  115.610869] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.622Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  115.614688] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.626Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  115.618303] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.630Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  115.622177] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.634Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1279vm-test-run-nixbot-gitlab> gitlab # [  115.625697] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.637Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1280vm-test-run-nixbot-gitlab> gitlab # [  115.629555] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.641Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1281vm-test-run-nixbot-gitlab> gitlab # [  115.633343] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.645Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1282vm-test-run-nixbot-gitlab> gitlab # [  115.637495] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.649Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1283vm-test-run-nixbot-gitlab> gitlab # [  115.641180] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.653Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1284vm-test-run-nixbot-gitlab> gitlab # [  115.644934] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.656Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1285vm-test-run-nixbot-gitlab> gitlab # [  115.648539] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.660Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1286vm-test-run-nixbot-gitlab> gitlab # [  115.683051] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.694Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1287vm-test-run-nixbot-gitlab> gitlab # [  115.687043] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.698Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1288vm-test-run-nixbot-gitlab> gitlab # [  115.690804] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.702Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1289vm-test-run-nixbot-gitlab> gitlab # [  115.695527] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.707Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1290vm-test-run-nixbot-gitlab> gitlab # [  115.700968] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.713Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1291vm-test-run-nixbot-gitlab> gitlab # [  115.704921] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.717Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1292vm-test-run-nixbot-gitlab> gitlab # [  115.709726] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.721Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1293vm-test-run-nixbot-gitlab> gitlab # [  115.714110] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.726Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1294vm-test-run-nixbot-gitlab> gitlab # [  115.718553] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.730Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1295vm-test-run-nixbot-gitlab> gitlab # [  115.723359] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.735Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1296vm-test-run-nixbot-gitlab> gitlab # [  115.727372] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.739Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1297vm-test-run-nixbot-gitlab> gitlab # [  115.732131] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.744Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1298vm-test-run-nixbot-gitlab> gitlab # [  115.749884] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.761Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1299vm-test-run-nixbot-gitlab> gitlab # [  115.763510] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.775Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1300vm-test-run-nixbot-gitlab> gitlab # [  115.770461] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.782Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1301vm-test-run-nixbot-gitlab> gitlab # [  115.780982] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.793Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1302vm-test-run-nixbot-gitlab> gitlab # [  115.786314] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.798Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1303vm-test-run-nixbot-gitlab> gitlab # [  115.791417] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.803Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1304vm-test-run-nixbot-gitlab> gitlab # [  115.795975] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.808Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1305vm-test-run-nixbot-gitlab> gitlab # [  115.801174] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.813Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1306vm-test-run-nixbot-gitlab> gitlab # [  115.804191] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.816Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1307vm-test-run-nixbot-gitlab> gitlab # [  115.807287] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.819Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1308vm-test-run-nixbot-gitlab> gitlab # [  115.811075] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.823Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1309vm-test-run-nixbot-gitlab> gitlab # [  115.814405] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.826Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1310vm-test-run-nixbot-gitlab> gitlab # [  115.817782] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.829Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1311vm-test-run-nixbot-gitlab> gitlab # [  115.821324] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.833Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1312vm-test-run-nixbot-gitlab> gitlab # [  115.828557] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.840Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1313vm-test-run-nixbot-gitlab> gitlab # [  115.834690] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.846Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1314vm-test-run-nixbot-gitlab> gitlab # [  115.840916] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.853Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1315vm-test-run-nixbot-gitlab> gitlab # [  115.848283] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.860Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1316vm-test-run-nixbot-gitlab> gitlab # [  115.858444] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.870Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1317vm-test-run-nixbot-gitlab> gitlab # [  115.862069] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.873Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1318vm-test-run-nixbot-gitlab> gitlab # [  115.865273] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.877Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1319vm-test-run-nixbot-gitlab> gitlab # [  115.868476] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.880Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1320vm-test-run-nixbot-gitlab> gitlab # [  115.875973] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.888Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1321vm-test-run-nixbot-gitlab> gitlab # [  115.883065] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.895Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1322vm-test-run-nixbot-gitlab> gitlab # [  115.888937] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.901Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1323vm-test-run-nixbot-gitlab> gitlab # [  115.895525] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.907Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1324vm-test-run-nixbot-gitlab> gitlab # [  115.901473] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.913Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1325vm-test-run-nixbot-gitlab> gitlab # [  115.907564] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.919Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1326vm-test-run-nixbot-gitlab> gitlab # [  115.914415] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.926Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1327vm-test-run-nixbot-gitlab> gitlab # [  115.921132] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.933Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1328vm-test-run-nixbot-gitlab> gitlab # [  115.927832] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.939Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1329vm-test-run-nixbot-gitlab> gitlab # [  115.934984] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.947Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1330vm-test-run-nixbot-gitlab> gitlab # [  115.943122] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.955Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1331vm-test-run-nixbot-gitlab> gitlab # [  115.947873] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.959Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1332vm-test-run-nixbot-gitlab> gitlab # [  115.951769] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.963Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1333vm-test-run-nixbot-gitlab> gitlab # [  115.955833] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.967Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1334vm-test-run-nixbot-gitlab> gitlab # [  115.958608] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.970Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1335vm-test-run-nixbot-gitlab> gitlab # [  115.961795] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.973Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1336vm-test-run-nixbot-gitlab> gitlab # [  115.965170] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.977Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1337vm-test-run-nixbot-gitlab> gitlab # [  115.968276] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.980Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1338vm-test-run-nixbot-gitlab> gitlab # [  115.971433] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.983Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1339vm-test-run-nixbot-gitlab> gitlab # [  115.974733] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.986Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1340vm-test-run-nixbot-gitlab> gitlab # [  115.978184] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.990Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1341vm-test-run-nixbot-gitlab> gitlab # [  115.981175] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.993Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1342vm-test-run-nixbot-gitlab> gitlab # [  115.984459] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.996Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1343vm-test-run-nixbot-gitlab> gitlab # [  115.987624] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:30.999Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1344vm-test-run-nixbot-gitlab> gitlab # [  115.990922] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.003Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1345vm-test-run-nixbot-gitlab> gitlab # [  115.994480] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.006Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1346vm-test-run-nixbot-gitlab> gitlab # [  115.998175] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.010Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1347vm-test-run-nixbot-gitlab> gitlab # [  116.001498] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.013Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1348vm-test-run-nixbot-gitlab> gitlab # [  116.004846] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.016Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1349vm-test-run-nixbot-gitlab> gitlab # [  116.008156] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.020Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1350vm-test-run-nixbot-gitlab> gitlab # [  116.011815] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.023Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1351vm-test-run-nixbot-gitlab> gitlab # [  116.015150] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.027Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1352vm-test-run-nixbot-gitlab> gitlab # [  116.018353] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.030Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1353vm-test-run-nixbot-gitlab> gitlab # [  116.021498] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.033Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1354vm-test-run-nixbot-gitlab> gitlab # [  116.024521] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.036Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1355vm-test-run-nixbot-gitlab> gitlab # [  116.028053] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.040Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1356vm-test-run-nixbot-gitlab> gitlab # [  116.031292] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.043Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1357vm-test-run-nixbot-gitlab> gitlab # [  116.034374] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.046Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1358vm-test-run-nixbot-gitlab> gitlab # [  116.040118] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.052Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1359vm-test-run-nixbot-gitlab> gitlab # [  116.043821] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.055Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1360vm-test-run-nixbot-gitlab> gitlab # [  116.047251] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.059Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1361vm-test-run-nixbot-gitlab> gitlab # [  116.050362] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.062Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1362vm-test-run-nixbot-gitlab> gitlab # [  116.058953] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.071Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1363vm-test-run-nixbot-gitlab> gitlab # [  116.062322] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.074Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1364vm-test-run-nixbot-gitlab> gitlab # [  116.065537] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.077Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1365vm-test-run-nixbot-gitlab> gitlab # [  116.069682] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.081Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1366vm-test-run-nixbot-gitlab> gitlab # [  116.073566] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.085Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1367vm-test-run-nixbot-gitlab> gitlab # [  116.077449] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.089Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1368vm-test-run-nixbot-gitlab> gitlab # [  116.081034] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.093Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1369vm-test-run-nixbot-gitlab> gitlab # [  116.084177] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.096Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1370vm-test-run-nixbot-gitlab> gitlab # [  116.087328] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.099Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1371vm-test-run-nixbot-gitlab> gitlab # [  116.090899] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.103Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1372vm-test-run-nixbot-gitlab> gitlab # [  116.094214] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:31.106Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1373vm-test-run-nixbot-gitlab> gitlab # [  119.566507] gitlab-workhorse[1288]: time="2026-07-04T13:31:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1374vm-test-run-nixbot-gitlab> gitlab # [  129.569514] gitlab-workhorse[1288]: time="2026-07-04T13:31:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1375vm-test-run-nixbot-gitlab> gitlab # [  137.064721] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:52.075Z","message":"Booted Rails 7.2.3 application in production environment"}1376vm-test-run-nixbot-gitlab> gitlab # [  137.066591] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:52.076Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1377vm-test-run-nixbot-gitlab> gitlab # [  137.068484] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:52.076Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1378vm-test-run-nixbot-gitlab> gitlab # [  137.070193] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:52.076Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1379vm-test-run-nixbot-gitlab> gitlab # [  137.079468] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:52.091Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1380vm-test-run-nixbot-gitlab> gitlab # [  138.417217] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:53.428Z","message":"Cleaning working queues"}1381vm-test-run-nixbot-gitlab> gitlab # [  138.426198] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:53.437Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"454cec037d48c49152e66fe0","created_at":"2026-07-04T13:30:57.753Z","trace_propagation_headers":{"sentry-trace":"31d531ef64af4b86a547fc9651828008-39aca904aaa94dd8","baggage":"sentry-trace_id=31d531ef64af4b86a547fc9651828008,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6270262ddd00952179c34ac6bee282e2","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T13:30:57.763Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-454cec037d48c49152e66fe0: start","job_status":"start","queue_duration_s":55.674164,"scheduling_latency_s":55.674252}1382vm-test-run-nixbot-gitlab> gitlab # [  138.703583] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:31:53.714Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"454cec037d48c49152e66fe0","created_at":"2026-07-04T13:30:57.753Z","trace_propagation_headers":{"sentry-trace":"31d531ef64af4b86a547fc9651828008-39aca904aaa94dd8","baggage":"sentry-trace_id=31d531ef64af4b86a547fc9651828008,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6270262ddd00952179c34ac6bee282e2","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T13:30:57.763Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-454cec037d48c49152e66fe0: done: 0.277406 sec","job_status":"done","queue_duration_s":55.674164,"scheduling_latency_s":55.674252,"redis_calls":31,"redis_duration_s":0.015498999999999999,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.008903,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003809,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002787,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.106393411,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.277406,"completed_at":"2026-07-04T13:31:53.714Z","load_balancing_strategy":"primary","db_duration_s":0.080913,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1383vm-test-run-nixbot-gitlab> gitlab # [  139.070142] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.080Z","pid":2107,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1384vm-test-run-nixbot-gitlab> gitlab # [  139.072133] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.081Z","pid":2107,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1385vm-test-run-nixbot-gitlab> gitlab # [  139.074809] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.081Z","pid":2107,"message":"! #\u003cThread:0x00007dbe14163b68 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1386vm-test-run-nixbot-gitlab> gitlab # [  139.078211] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.081Z","pid":2107,"message":"! #\u003cThread:0x00007dbe143a4940 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1387vm-test-run-nixbot-gitlab> gitlab # [  139.080930] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.081Z","pid":2107,"message":"Use Ctrl-C to stop"}1388vm-test-run-nixbot-gitlab> gitlab # [  139.083780] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.081Z","pid":2107,"message":"Sending status to systemd every 1.0 sec"}1389vm-test-run-nixbot-gitlab> gitlab # [  139.476938] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.488Z","pid":2107,"message":"- Worker 0 (PID: 2647) booted in 0.32s, phase: 0"}1390vm-test-run-nixbot-gitlab> gitlab # [  139.507160] bundle[2107]: {"timestamp":"2026-07-04T13:31:54.518Z","pid":2107,"message":"- Worker 1 (PID: 2651) booted in 0.34s, phase: 0"}1391vm-test-run-nixbot-gitlab> gitlab # [  139.514782] systemd[1]: Started gitlab.service.1392vm-test-run-nixbot-gitlab> gitlab # [  139.519769] systemd[1]: Reached target Common target for all GitLab services..1393vm-test-run-nixbot-gitlab> gitlab # [  139.527457] systemd[1]: Starting setup-gitlab.service...1394vm-test-run-nixbot-gitlab> gitlab # [  139.545554] setup-gitlab-start[2679]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1395vm-test-run-nixbot-gitlab> gitlab # [  141.897856] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:31:56 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 23261396vm-test-run-nixbot-gitlab> gitlab # [  141.909521] setup-gitlab-start[2702]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1397vm-test-run-nixbot-gitlab> gitlab # [  141.919797] setup-gitlab-start[2703]: ++ jq -r .access_token1398vm-test-run-nixbot-gitlab> gitlab # [  142.958766] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:31:57 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 10261399vm-test-run-nixbot-gitlab> gitlab # [  142.968879] setup-gitlab-start[2679]: + BEARER=348aa883e9ec738a2e684e7d73b6de5f67a3b2238f70263520fcd29a2e1a4e281400vm-test-run-nixbot-gitlab> gitlab # [  142.974693] setup-gitlab-start[2679]: + AUTH='Authorization: Bearer 348aa883e9ec738a2e684e7d73b6de5f67a3b2238f70263520fcd29a2e1a4e28'1401vm-test-run-nixbot-gitlab> gitlab # [  142.981501] setup-gitlab-start[2679]: + curl -fs -X PUT -H 'Authorization: Bearer 348aa883e9ec738a2e684e7d73b6de5f67a3b2238f70263520fcd29a2e1a4e28' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1402vm-test-run-nixbot-gitlab> gitlab # [  143.744120] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:31:58 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 7511403vm-test-run-nixbot-gitlab> gitlab # [  143.747526] setup-gitlab-start[2679]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1404vm-test-run-nixbot-gitlab> gitlab # [  143.764131] runuser[2721]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1405vm-test-run-nixbot-gitlab> gitlab # [  144.123600] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1406vm-test-run-nixbot-gitlab> gitlab # [  144.132421] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1407vm-test-run-nixbot-gitlab> gitlab # [  144.141268] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1408vm-test-run-nixbot-gitlab> gitlab # [  144.708787] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1409vm-test-run-nixbot-gitlab> gitlab # [  144.717572] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1410vm-test-run-nixbot-gitlab> gitlab # [  144.726191] setup-gitlab-start[2722]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1411vm-test-run-nixbot-gitlab> gitlab # [  151.779922] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:06.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ee60fc484162378ecfe352bf","created_at":"2026-07-04T13:32:06.751Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ff359868b22800935fad8483dc97b91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:06.787Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ee60fc484162378ecfe352bf: start","job_status":"start","queue_duration_s":0.002856,"scheduling_latency_s":0.002903}1412vm-test-run-nixbot-gitlab> gitlab # [  152.393165] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ee60fc484162378ecfe352bf","created_at":"2026-07-04T13:32:06.751Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ff359868b22800935fad8483dc97b91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:06.787Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ee60fc484162378ecfe352bf: done: 0.613811 sec","job_status":"done","queue_duration_s":0.002856,"scheduling_latency_s":0.002903,"redis_calls":12,"redis_duration_s":0.028364,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003409,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003016,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.228279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002212572,"exclusive_lock_hold_duration_s":0.44790972,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.613811,"completed_at":"2026-07-04T13:32:07.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413vm-test-run-nixbot-gitlab> gitlab # [  152.467898] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.479Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3602a387c84eec4a269294b0","created_at":"2026-07-04T13:32:06.843Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"de9d4bf511db65c97150851cc05c993c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:06.862Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3602a387c84eec4a269294b0: start","job_status":"start","queue_duration_s":0.617066,"scheduling_latency_s":0.617153}1414vm-test-run-nixbot-gitlab> gitlab # [  152.523824] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.535Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"3602a387c84eec4a269294b0","created_at":"2026-07-04T13:32:06.843Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"de9d4bf511db65c97150851cc05c993c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:06.862Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-3602a387c84eec4a269294b0: done: 0.056383 sec","job_status":"done","queue_duration_s":0.617066,"scheduling_latency_s":0.617153,"redis_calls":10,"redis_duration_s":0.010043999999999999,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001349,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001097625,"duration_s":0.056383,"completed_at":"2026-07-04T13:32:07.535Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415vm-test-run-nixbot-gitlab> gitlab # [  152.602519] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ae9ddb614c014dacc7bea268","created_at":"2026-07-04T13:32:06.899Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f7eea4c4152f475b497cd8820b6792","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T13:32:06.979Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ae9ddb614c014dacc7bea268: start","job_status":"start","queue_duration_s":0.634845,"scheduling_latency_s":0.634885}1416vm-test-run-nixbot-gitlab> gitlab # [  152.638979] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.650Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1417vm-test-run-nixbot-gitlab> gitlab # [  152.661745] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"ae9ddb614c014dacc7bea268","created_at":"2026-07-04T13:32:06.899Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53f7eea4c4152f475b497cd8820b6792","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T13:32:06.979Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-ae9ddb614c014dacc7bea268: done: 0.05917 sec","job_status":"done","queue_duration_s":0.634845,"scheduling_latency_s":0.634885,"redis_calls":10,"redis_duration_s":0.020666,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001167,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004015,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001085613,"duration_s":0.05917,"completed_at":"2026-07-04T13:32:07.673Z","load_balancing_strategy":"primary","db_duration_s":0.001026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [  152.698727] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"b01da2aa7d8574780e6b229a","created_at":"2026-07-04T13:32:06.989Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.002Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-b01da2aa7d8574780e6b229a: start","job_status":"start","queue_duration_s":0.680027,"scheduling_latency_s":0.680091}1419vm-test-run-nixbot-gitlab> gitlab # [  152.788530] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"b01da2aa7d8574780e6b229a","created_at":"2026-07-04T13:32:06.989Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.002Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-b01da2aa7d8574780e6b229a: done: 0.117207 sec","job_status":"done","queue_duration_s":0.680027,"scheduling_latency_s":0.680091,"redis_calls":65,"redis_duration_s":0.033181,"redis_read_bytes":114,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001706,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.001534,"redis_queues_read_bytes":3,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.028565,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515988,"duration_s":0.117207,"completed_at":"2026-07-04T13:32:07.799Z","load_balancing_strategy":"primary","db_duration_s":0.001545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420vm-test-run-nixbot-gitlab> gitlab # [  152.790534] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf3ad5dd8fee07a1c42bd496","created_at":"2026-07-04T13:32:07.007Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67f041dcefef6854aca78ba2a52904db","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.015Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf3ad5dd8fee07a1c42bd496: start","job_status":"start","queue_duration_s":0.785239,"scheduling_latency_s":0.785263}1421vm-test-run-nixbot-gitlab> gitlab # [  152.816469] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"cf3ad5dd8fee07a1c42bd496","created_at":"2026-07-04T13:32:07.007Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67f041dcefef6854aca78ba2a52904db","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.015Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-cf3ad5dd8fee07a1c42bd496: done: 0.027237 sec","job_status":"done","queue_duration_s":0.785239,"scheduling_latency_s":0.785263,"redis_calls":12,"redis_duration_s":0.0033439999999999998,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000458997,"exclusive_lock_hold_duration_s":0.0073984260000000005,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.027237,"completed_at":"2026-07-04T13:32:07.828Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422vm-test-run-nixbot-gitlab> gitlab # [  152.880077] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.829Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d0b151f42d6a7c88dfea621e","created_at":"2026-07-04T13:32:07.022Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6748cc84929494f7f6ad21b251c7a213","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.034Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d0b151f42d6a7c88dfea621e: start","job_status":"start","queue_duration_s":0.795018,"scheduling_latency_s":0.795038}1423vm-test-run-nixbot-gitlab> gitlab # [  152.952252] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d0b151f42d6a7c88dfea621e","created_at":"2026-07-04T13:32:07.022Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6748cc84929494f7f6ad21b251c7a213","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.034Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d0b151f42d6a7c88dfea621e: done: 0.030599 sec","job_status":"done","queue_duration_s":0.795018,"scheduling_latency_s":0.795038,"redis_calls":16,"redis_duration_s":0.003861,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001939,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346133,"duration_s":0.030599,"completed_at":"2026-07-04T13:32:07.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424vm-test-run-nixbot-gitlab> gitlab # [  153.027619] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.861Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7c55c8e05d0929e5e4629b2f","created_at":"2026-07-04T13:32:07.053Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2104701d7ba229e287dd05ba4bf2321","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.065Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7c55c8e05d0929e5e4629b2f: start","job_status":"start","queue_duration_s":0.796681,"scheduling_latency_s":0.796701}1425vm-test-run-nixbot-gitlab> gitlab # [  153.028800] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.873Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1426vm-test-run-nixbot-gitlab> gitlab # [  153.030320] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7c55c8e05d0929e5e4629b2f","created_at":"2026-07-04T13:32:07.053Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2104701d7ba229e287dd05ba4bf2321","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.065Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7c55c8e05d0929e5e4629b2f: done: 0.017905 sec","job_status":"done","queue_duration_s":0.796681,"scheduling_latency_s":0.796701,"redis_calls":10,"redis_duration_s":0.002327,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001422,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.017905,"completed_at":"2026-07-04T13:32:07.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000923,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427vm-test-run-nixbot-gitlab> gitlab # [  153.113976] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7010614f32967ad1be63c9ff","created_at":"2026-07-04T13:32:07.083Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"803e7325b87c3198a1e48c87b3d300f6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.088Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7010614f32967ad1be63c9ff: start","job_status":"start","queue_duration_s":0.79227,"scheduling_latency_s":0.792289}1428vm-test-run-nixbot-gitlab> gitlab # [  153.115224] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7010614f32967ad1be63c9ff","created_at":"2026-07-04T13:32:07.083Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"803e7325b87c3198a1e48c87b3d300f6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.088Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7010614f32967ad1be63c9ff: done: 0.021692 sec","job_status":"done","queue_duration_s":0.79227,"scheduling_latency_s":0.792289,"redis_calls":10,"redis_duration_s":0.002345,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340266,"duration_s":0.021692,"completed_at":"2026-07-04T13:32:07.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429vm-test-run-nixbot-gitlab> gitlab # [  153.199249] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f63cd636787089cedf2c8cb2","created_at":"2026-07-04T13:32:07.103Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05886e56d9d450ac6f654144d40bc45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.109Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f63cd636787089cedf2c8cb2: start","job_status":"start","queue_duration_s":0.794341,"scheduling_latency_s":0.794361}1430vm-test-run-nixbot-gitlab> gitlab # [  153.200712] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.915Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1431vm-test-run-nixbot-gitlab> gitlab # [  153.201817] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f63cd636787089cedf2c8cb2","created_at":"2026-07-04T13:32:07.103Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05886e56d9d450ac6f654144d40bc45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.109Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f63cd636787089cedf2c8cb2: done: 0.017056 sec","job_status":"done","queue_duration_s":0.794341,"scheduling_latency_s":0.794361,"redis_calls":10,"redis_duration_s":0.002303,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340546,"duration_s":0.017056,"completed_at":"2026-07-04T13:32:07.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [  153.288244] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3cf471a64862c3d32170bf9a","created_at":"2026-07-04T13:32:07.122Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b2b0930c04b82e4daf62c02a027a44d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.130Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3cf471a64862c3d32170bf9a: start","job_status":"start","queue_duration_s":0.79177,"scheduling_latency_s":0.79179}1433vm-test-run-nixbot-gitlab> gitlab # [  153.289627] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.933Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1434vm-test-run-nixbot-gitlab> gitlab # [  153.290848] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3cf471a64862c3d32170bf9a","created_at":"2026-07-04T13:32:07.122Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b2b0930c04b82e4daf62c02a027a44d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.130Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3cf471a64862c3d32170bf9a: done: 0.017094 sec","job_status":"done","queue_duration_s":0.79177,"scheduling_latency_s":0.79179,"redis_calls":10,"redis_duration_s":0.002356,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336076,"duration_s":0.017094,"completed_at":"2026-07-04T13:32:07.939Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435vm-test-run-nixbot-gitlab> gitlab # [  153.360628] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:07.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9d987fdd5ee86a796cca5994","created_at":"2026-07-04T13:32:07.139Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.144Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9d987fdd5ee86a796cca5994: start","job_status":"start","queue_duration_s":0.796997,"scheduling_latency_s":0.797061}1436vm-test-run-nixbot-gitlab> gitlab # [  153.362047] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.047Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9d987fdd5ee86a796cca5994","created_at":"2026-07-04T13:32:07.139Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.144Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9d987fdd5ee86a796cca5994: done: 0.105763 sec","job_status":"done","queue_duration_s":0.796997,"scheduling_latency_s":0.797061,"redis_calls":71,"redis_duration_s":0.017406,"redis_read_bytes":121,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002325,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000762,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013166,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415695,"duration_s":0.105763,"completed_at":"2026-07-04T13:32:08.046Z","load_balancing_strategy":"primary","db_duration_s":0.005598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437vm-test-run-nixbot-gitlab> gitlab # [  153.431203] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"be65277c05914e3e60f2a70f","created_at":"2026-07-04T13:32:07.157Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c9f0f4f85aadc9febbf1e550c324282","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.164Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-be65277c05914e3e60f2a70f: start","job_status":"start","queue_duration_s":0.883627,"scheduling_latency_s":0.883649}1438vm-test-run-nixbot-gitlab> gitlab # [  153.500317] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"be65277c05914e3e60f2a70f","created_at":"2026-07-04T13:32:07.157Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c9f0f4f85aadc9febbf1e550c324282","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.164Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-be65277c05914e3e60f2a70f: done: 0.020106 sec","job_status":"done","queue_duration_s":0.883627,"scheduling_latency_s":0.883649,"redis_calls":12,"redis_duration_s":0.002955,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000929,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001349,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386921,"duration_s":0.020106,"completed_at":"2026-07-04T13:32:08.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439vm-test-run-nixbot-gitlab> gitlab # [  153.578554] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c92ae5f415f14a7227a0ad56","created_at":"2026-07-04T13:32:07.177Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c27d2417e92962ef9b8094efcfd9a40a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.183Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c92ae5f415f14a7227a0ad56: start","job_status":"start","queue_duration_s":0.886368,"scheduling_latency_s":0.886389}1440vm-test-run-nixbot-gitlab> gitlab # [  153.579974] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"c92ae5f415f14a7227a0ad56","created_at":"2026-07-04T13:32:07.177Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c27d2417e92962ef9b8094efcfd9a40a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.183Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-c92ae5f415f14a7227a0ad56: done: 0.026512 sec","job_status":"done","queue_duration_s":0.886368,"scheduling_latency_s":0.886389,"redis_calls":10,"redis_duration_s":0.002486,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377981,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.026512,"completed_at":"2026-07-04T13:32:08.096Z","load_balancing_strategy":"primary","db_duration_s":0.006577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441vm-test-run-nixbot-gitlab> gitlab # [  153.605410] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"12ab6bd82ee72137b71188a1","created_at":"2026-07-04T13:32:07.193Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b54d291276047a61c23ec2d5e22eaeb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.200Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-12ab6bd82ee72137b71188a1: start","job_status":"start","queue_duration_s":0.898139,"scheduling_latency_s":0.89817}1442vm-test-run-nixbot-gitlab> gitlab # [  153.690294] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"12ab6bd82ee72137b71188a1","created_at":"2026-07-04T13:32:07.193Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b54d291276047a61c23ec2d5e22eaeb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.200Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-12ab6bd82ee72137b71188a1: done: 0.018688 sec","job_status":"done","queue_duration_s":0.898139,"scheduling_latency_s":0.89817,"redis_calls":10,"redis_duration_s":0.002187,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000461511,"duration_s":0.018688,"completed_at":"2026-07-04T13:32:08.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443vm-test-run-nixbot-gitlab> gitlab # [  153.694864] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4d5764c8de5ddddc952a9068","created_at":"2026-07-04T13:32:07.213Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c23c4e5af69131161235c7ee46c7027","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T13:32:07.222Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4d5764c8de5ddddc952a9068: start","job_status":"start","queue_duration_s":0.896329,"scheduling_latency_s":0.896359}1444vm-test-run-nixbot-gitlab> gitlab # [  153.716248] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.130Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1445vm-test-run-nixbot-gitlab> gitlab # [  153.721890] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4d5764c8de5ddddc952a9068","created_at":"2026-07-04T13:32:07.213Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c23c4e5af69131161235c7ee46c7027","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T13:32:07.222Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4d5764c8de5ddddc952a9068: done: 0.018481 sec","job_status":"done","queue_duration_s":0.896329,"scheduling_latency_s":0.896359,"redis_calls":10,"redis_duration_s":0.002449,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350044,"duration_s":0.018481,"completed_at":"2026-07-04T13:32:08.136Z","load_balancing_strategy":"primary","db_duration_s":0.000343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [  153.788721] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f282a1f28482bf5d5c3b2994","created_at":"2026-07-04T13:32:07.235Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b51341b4085cc5eb4b95bfca7325f0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.244Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f282a1f28482bf5d5c3b2994: start","job_status":"start","queue_duration_s":0.894377,"scheduling_latency_s":0.894397}1447vm-test-run-nixbot-gitlab> gitlab # [  153.812292] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.149Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1448vm-test-run-nixbot-gitlab> gitlab # [  153.818219] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f282a1f28482bf5d5c3b2994","created_at":"2026-07-04T13:32:07.235Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b51341b4085cc5eb4b95bfca7325f0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.244Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f282a1f28482bf5d5c3b2994: done: 0.016751 sec","job_status":"done","queue_duration_s":0.894377,"scheduling_latency_s":0.894397,"redis_calls":10,"redis_duration_s":0.00225,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000329931,"duration_s":0.016751,"completed_at":"2026-07-04T13:32:08.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449vm-test-run-nixbot-gitlab> gitlab # [  153.919315] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2976a26e3652be6022490779","created_at":"2026-07-04T13:32:07.266Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a58bbd2b51b9357d4f3c2008607a0984","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T13:32:07.275Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2976a26e3652be6022490779: start","job_status":"start","queue_duration_s":0.881423,"scheduling_latency_s":0.881443}1450vm-test-run-nixbot-gitlab> gitlab # [  153.920731] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2976a26e3652be6022490779","created_at":"2026-07-04T13:32:07.266Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a58bbd2b51b9357d4f3c2008607a0984","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T13:32:07.275Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2976a26e3652be6022490779: done: 0.025092 sec","job_status":"done","queue_duration_s":0.881423,"scheduling_latency_s":0.881443,"redis_calls":10,"redis_duration_s":0.002193,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000328254,"duration_s":0.025092,"completed_at":"2026-07-04T13:32:08.181Z","load_balancing_strategy":"primary","db_duration_s":0.00257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451vm-test-run-nixbot-gitlab> gitlab # [  154.001068] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6ac397799d99e548bb8f5c7c","created_at":"2026-07-04T13:32:07.301Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c572cd6da69e528086e78d3d5ce9563","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.307Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6ac397799d99e548bb8f5c7c: start","job_status":"start","queue_duration_s":0.87533,"scheduling_latency_s":0.875351}1452vm-test-run-nixbot-gitlab> gitlab # [  154.002413] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6ac397799d99e548bb8f5c7c","created_at":"2026-07-04T13:32:07.301Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c572cd6da69e528086e78d3d5ce9563","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.307Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6ac397799d99e548bb8f5c7c: done: 0.021013 sec","job_status":"done","queue_duration_s":0.87533,"scheduling_latency_s":0.875351,"redis_calls":10,"redis_duration_s":0.002334,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000324064,"duration_s":0.021013,"completed_at":"2026-07-04T13:32:08.203Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453vm-test-run-nixbot-gitlab> gitlab # [  154.072302] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dc0731c1620238e3abb3dc25","created_at":"2026-07-04T13:32:07.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee9f7d465f328c6a9af528c37fcaa270","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.323Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dc0731c1620238e3abb3dc25: start","job_status":"start","queue_duration_s":0.882121,"scheduling_latency_s":0.882141}1454vm-test-run-nixbot-gitlab> gitlab # [  154.141249] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"dc0731c1620238e3abb3dc25","created_at":"2026-07-04T13:32:07.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee9f7d465f328c6a9af528c37fcaa270","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.323Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-dc0731c1620238e3abb3dc25: done: 0.020765 sec","job_status":"done","queue_duration_s":0.882121,"scheduling_latency_s":0.882141,"redis_calls":12,"redis_duration_s":0.003087,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000498,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000480229,"duration_s":0.020765,"completed_at":"2026-07-04T13:32:08.226Z","load_balancing_strategy":"primary","db_duration_s":0.002882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455vm-test-run-nixbot-gitlab> gitlab # [  154.145947] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c9196404ae3c91e0bf68c6d4","created_at":"2026-07-04T13:32:07.336Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21d887fdb4f966a9960afcf6ba3526cd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.344Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c9196404ae3c91e0bf68c6d4: start","job_status":"start","queue_duration_s":0.882699,"scheduling_latency_s":0.882719}1456vm-test-run-nixbot-gitlab> gitlab # [  154.169107] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"c9196404ae3c91e0bf68c6d4","created_at":"2026-07-04T13:32:07.336Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21d887fdb4f966a9960afcf6ba3526cd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.344Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-c9196404ae3c91e0bf68c6d4: done: 0.029967 sec","job_status":"done","queue_duration_s":0.882699,"scheduling_latency_s":0.882719,"redis_calls":12,"redis_duration_s":0.00277,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000439162,"exclusive_lock_hold_duration_s":0.0031671620000000003,"duration_s":0.029967,"completed_at":"2026-07-04T13:32:08.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457vm-test-run-nixbot-gitlab> gitlab # [  154.268944] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"71271daf859e2868470620d9","created_at":"2026-07-04T13:32:07.359Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf674bd69711798962443b94534d203","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.365Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-71271daf859e2868470620d9: start","job_status":"start","queue_duration_s":0.896876,"scheduling_latency_s":0.896898}1458vm-test-run-nixbot-gitlab> gitlab # [  154.270207] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"71271daf859e2868470620d9","created_at":"2026-07-04T13:32:07.359Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf674bd69711798962443b94534d203","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.365Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-71271daf859e2868470620d9: done: 0.16045 sec","job_status":"done","queue_duration_s":0.896876,"scheduling_latency_s":0.896898,"redis_calls":14,"redis_duration_s":0.003195,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000816,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000438604,"exclusive_lock_hold_duration_s":0.140073948,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.16045,"completed_at":"2026-07-04T13:32:08.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459vm-test-run-nixbot-gitlab> gitlab # [  154.361976] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"98a93bfa7abfec0eb1cbfdb6","created_at":"2026-07-04T13:32:07.378Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.386Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-98a93bfa7abfec0eb1cbfdb6: start","job_status":"start","queue_duration_s":1.037639,"scheduling_latency_s":1.03766}1460vm-test-run-nixbot-gitlab> gitlab # [  154.363148] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"98a93bfa7abfec0eb1cbfdb6","created_at":"2026-07-04T13:32:07.378Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.386Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-98a93bfa7abfec0eb1cbfdb6: done: 0.029361 sec","job_status":"done","queue_duration_s":1.037639,"scheduling_latency_s":1.03766,"redis_calls":19,"redis_duration_s":0.0047150000000000004,"redis_read_bytes":23,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000868,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000298,"redis_queues_read_bytes":4,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002651,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060706,"duration_s":0.029361,"completed_at":"2026-07-04T13:32:08.453Z","load_balancing_strategy":"primary","db_duration_s":0.001599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461vm-test-run-nixbot-gitlab> gitlab # [  154.417920] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3b546522adf6d189215e75e4","created_at":"2026-07-04T13:32:07.394Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44655c96ba71b89b343d9fe3f9ee1465","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.475Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3b546522adf6d189215e75e4: start","job_status":"start","queue_duration_s":0.978892,"scheduling_latency_s":0.978912}1462vm-test-run-nixbot-gitlab> gitlab # [  154.440862] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3b546522adf6d189215e75e4","created_at":"2026-07-04T13:32:07.394Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44655c96ba71b89b343d9fe3f9ee1465","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.475Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3b546522adf6d189215e75e4: done: 0.406054 sec","job_status":"done","queue_duration_s":0.978892,"scheduling_latency_s":0.978912,"redis_calls":574,"redis_duration_s":0.124766,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.12295,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.31946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355352,"duration_s":0.406054,"completed_at":"2026-07-04T13:32:08.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463vm-test-run-nixbot-gitlab> gitlab # [  154.508275] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bd9ab635080c1f9392d29f74","created_at":"2026-07-04T13:32:07.520Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"992df6a7d00af5fceabd721e63835ebd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.529Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bd9ab635080c1f9392d29f74: start","job_status":"start","queue_duration_s":1.333334,"scheduling_latency_s":1.333354}1464vm-test-run-nixbot-gitlab> gitlab # [  154.578253] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bd9ab635080c1f9392d29f74","created_at":"2026-07-04T13:32:07.520Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"992df6a7d00af5fceabd721e63835ebd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.529Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bd9ab635080c1f9392d29f74: done: 0.041627 sec","job_status":"done","queue_duration_s":1.333334,"scheduling_latency_s":1.333354,"redis_calls":14,"redis_duration_s":0.004222,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000897,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000453409,"exclusive_lock_hold_duration_s":0.019972092,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041627,"completed_at":"2026-07-04T13:32:08.904Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465vm-test-run-nixbot-gitlab> gitlab # [  154.601320] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3eeabb9c4da0bf1348f209c8","created_at":"2026-07-04T13:32:07.615Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"634874fd61a92cb1af4cc1d0b5d04927","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.643Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3eeabb9c4da0bf1348f209c8: start","job_status":"start","queue_duration_s":1.262045,"scheduling_latency_s":1.262066}1466vm-test-run-nixbot-gitlab> gitlab # [  154.624814] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"3eeabb9c4da0bf1348f209c8","created_at":"2026-07-04T13:32:07.615Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"634874fd61a92cb1af4cc1d0b5d04927","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.643Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-3eeabb9c4da0bf1348f209c8: done: 0.075905 sec","job_status":"done","queue_duration_s":1.262045,"scheduling_latency_s":1.262066,"redis_calls":13,"redis_duration_s":0.004633,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000728,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002393,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001512,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007833678,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.075905,"completed_at":"2026-07-04T13:32:08.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002954,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1467vm-test-run-nixbot-gitlab> gitlab # [  154.694259] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:08.983Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"83273b4630e24d7a849804fa","created_at":"2026-07-04T13:32:07.660Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebb04d4022b422071e8b77b8fe7978c8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.676Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-83273b4630e24d7a849804fa: start","job_status":"start","queue_duration_s":1.307219,"scheduling_latency_s":1.307244}1468vm-test-run-nixbot-gitlab> gitlab # [  154.764063] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"83273b4630e24d7a849804fa","created_at":"2026-07-04T13:32:07.660Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebb04d4022b422071e8b77b8fe7978c8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.676Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-83273b4630e24d7a849804fa: done: 0.02265 sec","job_status":"done","queue_duration_s":1.307219,"scheduling_latency_s":1.307244,"redis_calls":12,"redis_duration_s":0.0041340000000000005,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001432,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000573536,"exclusive_lock_hold_duration_s":0.004604775,"duration_s":0.02265,"completed_at":"2026-07-04T13:32:09.006Z","load_balancing_strategy":"primary","db_duration_s":0.002388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469vm-test-run-nixbot-gitlab> gitlab # [  154.779342] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d589b3ab944c3be7c3eb9ae2","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d589b3ab944c3be7c3eb9ae2: start","job_status":"start","queue_duration_s":1.218901,"scheduling_latency_s":1.218922}1470vm-test-run-nixbot-gitlab> gitlab # [  154.799456] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d589b3ab944c3be7c3eb9ae2","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d589b3ab944c3be7c3eb9ae2: done: 0.027587 sec","job_status":"done","queue_duration_s":1.218901,"scheduling_latency_s":1.218922,"redis_calls":13,"redis_duration_s":0.004693,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000638,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043553,"duration_s":0.027587,"completed_at":"2026-07-04T13:32:09.035Z","load_balancing_strategy":"primary","db_duration_s":0.005226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471vm-test-run-nixbot-gitlab> gitlab # [  154.868322] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"af1217b9d23b975092b8422c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-af1217b9d23b975092b8422c: start","job_status":"start","queue_duration_s":1.248081,"scheduling_latency_s":1.248101}1472vm-test-run-nixbot-gitlab> gitlab # [  154.880878] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"af1217b9d23b975092b8422c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-af1217b9d23b975092b8422c: done: 0.019272 sec","job_status":"done","queue_duration_s":1.248081,"scheduling_latency_s":1.248101,"redis_calls":10,"redis_duration_s":0.0031060000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019272,"completed_at":"2026-07-04T13:32:09.056Z","load_balancing_strategy":"primary","db_duration_s":0.004052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473vm-test-run-nixbot-gitlab> gitlab # [  154.955089] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e8c9f8e9cac9a96001ea0d8b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e8c9f8e9cac9a96001ea0d8b: start","job_status":"start","queue_duration_s":1.268967,"scheduling_latency_s":1.268987}1474vm-test-run-nixbot-gitlab> gitlab # [  154.960133] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e8c9f8e9cac9a96001ea0d8b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e8c9f8e9cac9a96001ea0d8b: done: 0.030172 sec","job_status":"done","queue_duration_s":1.268967,"scheduling_latency_s":1.268987,"redis_calls":10,"redis_duration_s":0.003322,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001844,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030172,"completed_at":"2026-07-04T13:32:09.088Z","load_balancing_strategy":"primary","db_duration_s":0.002495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475vm-test-run-nixbot-gitlab> gitlab # [  155.043338] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"31070c2924a333d2bd6db0db","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-31070c2924a333d2bd6db0db: start","job_status":"start","queue_duration_s":1.300577,"scheduling_latency_s":1.3006}1476vm-test-run-nixbot-gitlab> gitlab # [  155.044913] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"31070c2924a333d2bd6db0db","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-31070c2924a333d2bd6db0db: done: 0.019456 sec","job_status":"done","queue_duration_s":1.300577,"scheduling_latency_s":1.3006,"redis_calls":10,"redis_duration_s":0.002535,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001564,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019456,"completed_at":"2026-07-04T13:32:09.109Z","load_balancing_strategy":"primary","db_duration_s":0.001777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477vm-test-run-nixbot-gitlab> gitlab # [  155.118218] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9042fc5bf15c956b174e1158","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9042fc5bf15c956b174e1158: start","job_status":"start","queue_duration_s":1.321582,"scheduling_latency_s":1.321607}1478vm-test-run-nixbot-gitlab> gitlab # [  155.119802] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9042fc5bf15c956b174e1158","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9042fc5bf15c956b174e1158: done: 0.015974 sec","job_status":"done","queue_duration_s":1.321582,"scheduling_latency_s":1.321607,"redis_calls":10,"redis_duration_s":0.002212,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001382,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015974,"completed_at":"2026-07-04T13:32:09.126Z","load_balancing_strategy":"primary","db_duration_s":0.001545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479vm-test-run-nixbot-gitlab> gitlab # [  155.205301] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"140bb9bc1ce8ba70a25747f9","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-140bb9bc1ce8ba70a25747f9: start","job_status":"start","queue_duration_s":1.338761,"scheduling_latency_s":1.338781}1480vm-test-run-nixbot-gitlab> gitlab # [  155.207045] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"140bb9bc1ce8ba70a25747f9","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-140bb9bc1ce8ba70a25747f9: done: 0.015502 sec","job_status":"done","queue_duration_s":1.338761,"scheduling_latency_s":1.338781,"redis_calls":10,"redis_duration_s":0.0021799999999999996,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001303,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015502,"completed_at":"2026-07-04T13:32:09.143Z","load_balancing_strategy":"primary","db_duration_s":0.001478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481vm-test-run-nixbot-gitlab> gitlab # [  155.295080] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2d965fc8d9ec0c749556e8fc","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2d965fc8d9ec0c749556e8fc: start","job_status":"start","queue_duration_s":1.355493,"scheduling_latency_s":1.355513}1482vm-test-run-nixbot-gitlab> gitlab # [  155.296417] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.160Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2d965fc8d9ec0c749556e8fc","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2d965fc8d9ec0c749556e8fc: done: 0.01597 sec","job_status":"done","queue_duration_s":1.355493,"scheduling_latency_s":1.355513,"redis_calls":10,"redis_duration_s":0.0021809999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01597,"completed_at":"2026-07-04T13:32:09.160Z","load_balancing_strategy":"primary","db_duration_s":0.001742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483vm-test-run-nixbot-gitlab> gitlab # [  155.391055] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9baf935b5f5c8662b0d3e09","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9baf935b5f5c8662b0d3e09: start","job_status":"start","queue_duration_s":1.372694,"scheduling_latency_s":1.372715}1484vm-test-run-nixbot-gitlab> gitlab # [  155.392401] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9baf935b5f5c8662b0d3e09","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9baf935b5f5c8662b0d3e09: done: 0.015362 sec","job_status":"done","queue_duration_s":1.372694,"scheduling_latency_s":1.372715,"redis_calls":10,"redis_duration_s":0.0023150000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015362,"completed_at":"2026-07-04T13:32:09.177Z","load_balancing_strategy":"primary","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485vm-test-run-nixbot-gitlab> gitlab # [  155.416745] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f9a0968f33f0589393f602b3","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f9a0968f33f0589393f602b3: start","job_status":"start","queue_duration_s":1.389797,"scheduling_latency_s":1.389839}1486vm-test-run-nixbot-gitlab> gitlab # [  155.436357] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f9a0968f33f0589393f602b3","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f9a0968f33f0589393f602b3: done: 0.019451 sec","job_status":"done","queue_duration_s":1.389797,"scheduling_latency_s":1.389839,"redis_calls":10,"redis_duration_s":0.00228,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019451,"completed_at":"2026-07-04T13:32:09.198Z","load_balancing_strategy":"primary","db_duration_s":0.001912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487vm-test-run-nixbot-gitlab> gitlab # [  155.534284] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77790307398947bc3a2eacf2","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77790307398947bc3a2eacf2: start","job_status":"start","queue_duration_s":1.410853,"scheduling_latency_s":1.410873}1488vm-test-run-nixbot-gitlab> gitlab # [  155.535512] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77790307398947bc3a2eacf2","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77790307398947bc3a2eacf2: done: 0.015608 sec","job_status":"done","queue_duration_s":1.410853,"scheduling_latency_s":1.410873,"redis_calls":10,"redis_duration_s":0.002201,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015608,"completed_at":"2026-07-04T13:32:09.215Z","load_balancing_strategy":"primary","db_duration_s":0.001393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489vm-test-run-nixbot-gitlab> gitlab # [  155.628779] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9cc9b38b00b87791b069ba","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9cc9b38b00b87791b069ba: start","job_status":"start","queue_duration_s":1.427771,"scheduling_latency_s":1.427792}1490vm-test-run-nixbot-gitlab> gitlab # [  155.630233] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9cc9b38b00b87791b069ba","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9cc9b38b00b87791b069ba: done: 0.015522 sec","job_status":"done","queue_duration_s":1.427771,"scheduling_latency_s":1.427792,"redis_calls":10,"redis_duration_s":0.002307,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00139,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015522,"completed_at":"2026-07-04T13:32:09.232Z","load_balancing_strategy":"primary","db_duration_s":0.001526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491vm-test-run-nixbot-gitlab> gitlab # [  155.699466] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3fed6f8974a045a68e5812c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3fed6f8974a045a68e5812c: start","job_status":"start","queue_duration_s":1.444526,"scheduling_latency_s":1.444546}1492vm-test-run-nixbot-gitlab> gitlab # [  155.700923] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3fed6f8974a045a68e5812c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3fed6f8974a045a68e5812c: done: 0.015397 sec","job_status":"done","queue_duration_s":1.444526,"scheduling_latency_s":1.444546,"redis_calls":10,"redis_duration_s":0.002179,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015397,"completed_at":"2026-07-04T13:32:09.249Z","load_balancing_strategy":"primary","db_duration_s":0.001483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493vm-test-run-nixbot-gitlab> gitlab # [  155.771124] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"48ec6ae04239e9f733b5c618","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-48ec6ae04239e9f733b5c618: start","job_status":"start","queue_duration_s":1.461205,"scheduling_latency_s":1.461225}1494vm-test-run-nixbot-gitlab> gitlab # [  155.772260] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"48ec6ae04239e9f733b5c618","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-48ec6ae04239e9f733b5c618: done: 0.015334 sec","job_status":"done","queue_duration_s":1.461205,"scheduling_latency_s":1.461225,"redis_calls":10,"redis_duration_s":0.002193,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015334,"completed_at":"2026-07-04T13:32:09.265Z","load_balancing_strategy":"primary","db_duration_s":0.001461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495vm-test-run-nixbot-gitlab> gitlab # [  155.858279] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"34b9eabe56d1af158c5cb6a6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-34b9eabe56d1af158c5cb6a6: start","job_status":"start","queue_duration_s":1.47798,"scheduling_latency_s":1.478001}1496vm-test-run-nixbot-gitlab> gitlab # [  155.859857] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"34b9eabe56d1af158c5cb6a6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-34b9eabe56d1af158c5cb6a6: done: 0.015542 sec","job_status":"done","queue_duration_s":1.47798,"scheduling_latency_s":1.478001,"redis_calls":10,"redis_duration_s":0.002446,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015542,"completed_at":"2026-07-04T13:32:09.282Z","load_balancing_strategy":"primary","db_duration_s":0.001393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497vm-test-run-nixbot-gitlab> gitlab # [  155.931146] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cf71219cd50036f5b0838fe8","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cf71219cd50036f5b0838fe8: start","job_status":"start","queue_duration_s":1.494748,"scheduling_latency_s":1.494768}1498vm-test-run-nixbot-gitlab> gitlab # [  155.932488] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cf71219cd50036f5b0838fe8","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cf71219cd50036f5b0838fe8: done: 0.015262 sec","job_status":"done","queue_duration_s":1.494748,"scheduling_latency_s":1.494768,"redis_calls":10,"redis_duration_s":0.002162,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015262,"completed_at":"2026-07-04T13:32:09.299Z","load_balancing_strategy":"primary","db_duration_s":0.001478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499vm-test-run-nixbot-gitlab> gitlab # [  156.008183] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f7be321e493b33f680c0bd0f","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f7be321e493b33f680c0bd0f: start","job_status":"start","queue_duration_s":1.51129,"scheduling_latency_s":1.511311}1500vm-test-run-nixbot-gitlab> gitlab # [  156.009764] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f7be321e493b33f680c0bd0f","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f7be321e493b33f680c0bd0f: done: 0.015495 sec","job_status":"done","queue_duration_s":1.51129,"scheduling_latency_s":1.511311,"redis_calls":10,"redis_duration_s":0.002188,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001257,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015495,"completed_at":"2026-07-04T13:32:09.315Z","load_balancing_strategy":"primary","db_duration_s":0.001424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501vm-test-run-nixbot-gitlab> gitlab # [  156.091223] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d6dad3cf0e245bbb842c6ab6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d6dad3cf0e245bbb842c6ab6: start","job_status":"start","queue_duration_s":1.528393,"scheduling_latency_s":1.528412}1502vm-test-run-nixbot-gitlab> gitlab # [  156.092807] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d6dad3cf0e245bbb842c6ab6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d6dad3cf0e245bbb842c6ab6: done: 0.015565 sec","job_status":"done","queue_duration_s":1.528393,"scheduling_latency_s":1.528412,"redis_calls":10,"redis_duration_s":0.002288,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015565,"completed_at":"2026-07-04T13:32:09.333Z","load_balancing_strategy":"primary","db_duration_s":0.001474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503vm-test-run-nixbot-gitlab> gitlab # [  156.185506] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3f4c04b982ee943a2f36b34","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3f4c04b982ee943a2f36b34: start","job_status":"start","queue_duration_s":1.545236,"scheduling_latency_s":1.545257}1504vm-test-run-nixbot-gitlab> gitlab # [  156.186978] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b3f4c04b982ee943a2f36b34","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b3f4c04b982ee943a2f36b34: done: 0.015266 sec","job_status":"done","queue_duration_s":1.545236,"scheduling_latency_s":1.545257,"redis_calls":10,"redis_duration_s":0.002205,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015266,"completed_at":"2026-07-04T13:32:09.349Z","load_balancing_strategy":"primary","db_duration_s":0.001429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505vm-test-run-nixbot-gitlab> gitlab # [  156.205222] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bfa4e3b95f3c78ba3d05be1c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bfa4e3b95f3c78ba3d05be1c: start","job_status":"start","queue_duration_s":1.561819,"scheduling_latency_s":1.56184}1506vm-test-run-nixbot-gitlab> gitlab # [  156.225198] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bfa4e3b95f3c78ba3d05be1c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bfa4e3b95f3c78ba3d05be1c: done: 0.015264 sec","job_status":"done","queue_duration_s":1.561819,"scheduling_latency_s":1.56184,"redis_calls":10,"redis_duration_s":0.002176,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015264,"completed_at":"2026-07-04T13:32:09.366Z","load_balancing_strategy":"primary","db_duration_s":0.001359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507vm-test-run-nixbot-gitlab> gitlab # [  156.295413] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d6f60e8f26395b1e6643545a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d6f60e8f26395b1e6643545a: start","job_status":"start","queue_duration_s":1.578721,"scheduling_latency_s":1.578741}1508vm-test-run-nixbot-gitlab> gitlab # [  156.364047] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d6f60e8f26395b1e6643545a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d6f60e8f26395b1e6643545a: done: 0.015717 sec","job_status":"done","queue_duration_s":1.578721,"scheduling_latency_s":1.578741,"redis_calls":10,"redis_duration_s":0.0023759999999999996,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015717,"completed_at":"2026-07-04T13:32:09.383Z","load_balancing_strategy":"primary","db_duration_s":0.001533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509vm-test-run-nixbot-gitlab> gitlab # [  156.365789] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b4335d12cb68eff75d79cad4","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b4335d12cb68eff75d79cad4: start","job_status":"start","queue_duration_s":1.595662,"scheduling_latency_s":1.595682}1510vm-test-run-nixbot-gitlab> gitlab # [  156.433473] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b4335d12cb68eff75d79cad4","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b4335d12cb68eff75d79cad4: done: 0.019582 sec","job_status":"done","queue_duration_s":1.595662,"scheduling_latency_s":1.595682,"redis_calls":10,"redis_duration_s":0.002534,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001593,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019582,"completed_at":"2026-07-04T13:32:09.404Z","load_balancing_strategy":"primary","db_duration_s":0.002519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [  156.441286] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fcc15df20a13bf28a34aea80","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fcc15df20a13bf28a34aea80: start","job_status":"start","queue_duration_s":1.616845,"scheduling_latency_s":1.616869}1512vm-test-run-nixbot-gitlab> gitlab # [  156.513679] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fcc15df20a13bf28a34aea80","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fcc15df20a13bf28a34aea80: done: 0.016838 sec","job_status":"done","queue_duration_s":1.616845,"scheduling_latency_s":1.616869,"redis_calls":10,"redis_duration_s":0.002219,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016838,"completed_at":"2026-07-04T13:32:09.422Z","load_balancing_strategy":"primary","db_duration_s":0.001684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [  156.520205] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b08e3ae4efcadad356a75c08","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b08e3ae4efcadad356a75c08: start","job_status":"start","queue_duration_s":1.634894,"scheduling_latency_s":1.634914}1514vm-test-run-nixbot-gitlab> gitlab # [  156.539931] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b08e3ae4efcadad356a75c08","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b08e3ae4efcadad356a75c08: done: 0.0159 sec","job_status":"done","queue_duration_s":1.634894,"scheduling_latency_s":1.634914,"redis_calls":10,"redis_duration_s":0.002189,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001319,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0159,"completed_at":"2026-07-04T13:32:09.439Z","load_balancing_strategy":"primary","db_duration_s":0.001407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [  156.612314] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f86c9c2dd70a5967d0088ad","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f86c9c2dd70a5967d0088ad: start","job_status":"start","queue_duration_s":1.652016,"scheduling_latency_s":1.652043}1516vm-test-run-nixbot-gitlab> gitlab # [  156.682057] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f86c9c2dd70a5967d0088ad","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f86c9c2dd70a5967d0088ad: done: 0.015452 sec","job_status":"done","queue_duration_s":1.652016,"scheduling_latency_s":1.652043,"redis_calls":10,"redis_duration_s":0.002171,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015452,"completed_at":"2026-07-04T13:32:09.456Z","load_balancing_strategy":"primary","db_duration_s":0.001485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [  156.684166] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a67964c491c6c7a9783e303c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a67964c491c6c7a9783e303c: start","job_status":"start","queue_duration_s":1.668678,"scheduling_latency_s":1.6687}1518vm-test-run-nixbot-gitlab> gitlab # [  156.697099] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a67964c491c6c7a9783e303c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a67964c491c6c7a9783e303c: done: 0.015411 sec","job_status":"done","queue_duration_s":1.668678,"scheduling_latency_s":1.6687,"redis_calls":10,"redis_duration_s":0.002153,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015411,"completed_at":"2026-07-04T13:32:09.473Z","load_balancing_strategy":"primary","db_duration_s":0.0015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519vm-test-run-nixbot-gitlab> gitlab # [  156.814073] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e72e20fde1f4e1719ce1ce20","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e72e20fde1f4e1719ce1ce20: start","job_status":"start","queue_duration_s":1.685587,"scheduling_latency_s":1.685608}1520vm-test-run-nixbot-gitlab> gitlab # [  156.815400] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e72e20fde1f4e1719ce1ce20","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e72e20fde1f4e1719ce1ce20: done: 0.015419 sec","job_status":"done","queue_duration_s":1.685587,"scheduling_latency_s":1.685608,"redis_calls":10,"redis_duration_s":0.002191,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015419,"completed_at":"2026-07-04T13:32:09.490Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521vm-test-run-nixbot-gitlab> gitlab # [  156.885360] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8e843766800103711d4e2c6a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8e843766800103711d4e2c6a: start","job_status":"start","queue_duration_s":1.702715,"scheduling_latency_s":1.702736}1522vm-test-run-nixbot-gitlab> gitlab # [  156.886845] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8e843766800103711d4e2c6a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8e843766800103711d4e2c6a: done: 0.015435 sec","job_status":"done","queue_duration_s":1.702715,"scheduling_latency_s":1.702736,"redis_calls":10,"redis_duration_s":0.002143,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015435,"completed_at":"2026-07-04T13:32:09.507Z","load_balancing_strategy":"primary","db_duration_s":0.001446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523vm-test-run-nixbot-gitlab> gitlab # [  156.916359] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"aea89c28ff3673cf6c73c6b5","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-aea89c28ff3673cf6c73c6b5: start","job_status":"start","queue_duration_s":1.71947,"scheduling_latency_s":1.71949}1524vm-test-run-nixbot-gitlab> gitlab # [  156.936118] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"aea89c28ff3673cf6c73c6b5","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-aea89c28ff3673cf6c73c6b5: done: 0.015308 sec","job_status":"done","queue_duration_s":1.71947,"scheduling_latency_s":1.71949,"redis_calls":10,"redis_duration_s":0.002456,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001576,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015308,"completed_at":"2026-07-04T13:32:09.523Z","load_balancing_strategy":"primary","db_duration_s":0.001403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525vm-test-run-nixbot-gitlab> gitlab # [  157.017244] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b0c7174a8c0b5b7c7bda18e6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b0c7174a8c0b5b7c7bda18e6: start","job_status":"start","queue_duration_s":1.736004,"scheduling_latency_s":1.736024}1526vm-test-run-nixbot-gitlab> gitlab # [  157.018388] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b0c7174a8c0b5b7c7bda18e6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b0c7174a8c0b5b7c7bda18e6: done: 0.015516 sec","job_status":"done","queue_duration_s":1.736004,"scheduling_latency_s":1.736024,"redis_calls":10,"redis_duration_s":0.002177,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015516,"completed_at":"2026-07-04T13:32:09.540Z","load_balancing_strategy":"primary","db_duration_s":0.001488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527vm-test-run-nixbot-gitlab> gitlab # [  157.109385] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9d2d4bf7e29a316d2a2e7b56","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9d2d4bf7e29a316d2a2e7b56: start","job_status":"start","queue_duration_s":1.753051,"scheduling_latency_s":1.753072}1528vm-test-run-nixbot-gitlab> gitlab # [  157.110769] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9d2d4bf7e29a316d2a2e7b56","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9d2d4bf7e29a316d2a2e7b56: done: 0.015767 sec","job_status":"done","queue_duration_s":1.753051,"scheduling_latency_s":1.753072,"redis_calls":10,"redis_duration_s":0.002218,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015767,"completed_at":"2026-07-04T13:32:09.557Z","load_balancing_strategy":"primary","db_duration_s":0.001654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529vm-test-run-nixbot-gitlab> gitlab # [  157.180297] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cf356b24b35dd0a25f243f05","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cf356b24b35dd0a25f243f05: start","job_status":"start","queue_duration_s":1.770047,"scheduling_latency_s":1.770068}1530vm-test-run-nixbot-gitlab> gitlab # [  157.181427] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cf356b24b35dd0a25f243f05","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cf356b24b35dd0a25f243f05: done: 0.015634 sec","job_status":"done","queue_duration_s":1.770047,"scheduling_latency_s":1.770068,"redis_calls":10,"redis_duration_s":0.002173,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001343,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015634,"completed_at":"2026-07-04T13:32:09.574Z","load_balancing_strategy":"primary","db_duration_s":0.001442,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [  157.279988] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:09.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c24625cd5ef626603f1d148a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c24625cd5ef626603f1d148a: start","job_status":"start","queue_duration_s":1.786903,"scheduling_latency_s":1.786923}1532vm-test-run-nixbot-gitlab> gitlab # [  157.585172] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3047011024 wd_nsec: 30470095651533vm-test-run-nixbot-gitlab> gitlab # [  157.340449] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c24625cd5ef626603f1d148a","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c24625cd5ef626603f1d148a: done: 2.776216 sec","job_status":"done","queue_duration_s":1.786903,"scheduling_latency_s":1.786923,"redis_calls":10,"redis_duration_s":0.0025399999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001392,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":2.776216,"completed_at":"2026-07-04T13:32:12.352Z","load_balancing_strategy":"primary","db_duration_s":0.001382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [  157.402672] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1962fca2997731ddceed6f6b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1962fca2997731ddceed6f6b: start","job_status":"start","queue_duration_s":4.625306,"scheduling_latency_s":4.625342}1535vm-test-run-nixbot-gitlab> gitlab # [  157.440833] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1962fca2997731ddceed6f6b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1962fca2997731ddceed6f6b: done: 0.038215 sec","job_status":"done","queue_duration_s":4.625306,"scheduling_latency_s":4.625342,"redis_calls":10,"redis_duration_s":0.002447,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001464,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038215,"completed_at":"2026-07-04T13:32:12.452Z","load_balancing_strategy":"primary","db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536vm-test-run-nixbot-gitlab> gitlab # [  157.503769] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"53b9a4520bd1a95d93a96e29","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-53b9a4520bd1a95d93a96e29: start","job_status":"start","queue_duration_s":4.726074,"scheduling_latency_s":4.726234}1537vm-test-run-nixbot-gitlab> gitlab # [  157.546950] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"53b9a4520bd1a95d93a96e29","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-53b9a4520bd1a95d93a96e29: done: 0.043152 sec","job_status":"done","queue_duration_s":4.726074,"scheduling_latency_s":4.726234,"redis_calls":10,"redis_duration_s":0.005857,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002363,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003494,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043152,"completed_at":"2026-07-04T13:32:12.558Z","load_balancing_strategy":"primary","db_duration_s":0.003127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538vm-test-run-nixbot-gitlab> gitlab # [  157.610835] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26fe0377b6ef2471bc3bd583","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26fe0377b6ef2471bc3bd583: start","job_status":"start","queue_duration_s":4.833005,"scheduling_latency_s":4.833092}1539vm-test-run-nixbot-gitlab> gitlab # [  157.658871] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26fe0377b6ef2471bc3bd583","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26fe0377b6ef2471bc3bd583: done: 0.04786 sec","job_status":"done","queue_duration_s":4.833005,"scheduling_latency_s":4.833092,"redis_calls":10,"redis_duration_s":0.006547,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003875,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04786,"completed_at":"2026-07-04T13:32:12.669Z","load_balancing_strategy":"primary","db_duration_s":0.004512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540vm-test-run-nixbot-gitlab> gitlab # [  157.723764] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4843782a05f3e44cbc3aaf6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4843782a05f3e44cbc3aaf6: start","job_status":"start","queue_duration_s":4.945974,"scheduling_latency_s":4.946061}1541vm-test-run-nixbot-gitlab> gitlab # [  157.768501] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4843782a05f3e44cbc3aaf6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4843782a05f3e44cbc3aaf6: done: 0.044525 sec","job_status":"done","queue_duration_s":4.945974,"scheduling_latency_s":4.946061,"redis_calls":10,"redis_duration_s":0.006586,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004112,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044525,"completed_at":"2026-07-04T13:32:12.779Z","load_balancing_strategy":"primary","db_duration_s":0.004361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542vm-test-run-nixbot-gitlab> gitlab # [  157.832816] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c7db0cb1d363d316725143f1","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c7db0cb1d363d316725143f1: start","job_status":"start","queue_duration_s":5.05498,"scheduling_latency_s":5.055074}1543vm-test-run-nixbot-gitlab> gitlab # [  157.877136] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c7db0cb1d363d316725143f1","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c7db0cb1d363d316725143f1: done: 0.044313 sec","job_status":"done","queue_duration_s":5.05498,"scheduling_latency_s":5.055074,"redis_calls":10,"redis_duration_s":0.006307,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00258,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044313,"completed_at":"2026-07-04T13:32:12.888Z","load_balancing_strategy":"primary","db_duration_s":0.004411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544vm-test-run-nixbot-gitlab> gitlab # [  157.942768] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"75919d147f128dd61c7580e0","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-75919d147f128dd61c7580e0: start","job_status":"start","queue_duration_s":5.163924,"scheduling_latency_s":5.163992}1545vm-test-run-nixbot-gitlab> gitlab # [  157.986855] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:12.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"75919d147f128dd61c7580e0","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-75919d147f128dd61c7580e0: done: 0.045179 sec","job_status":"done","queue_duration_s":5.163924,"scheduling_latency_s":5.163992,"redis_calls":10,"redis_duration_s":0.006428,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00376,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045179,"completed_at":"2026-07-04T13:32:12.998Z","load_balancing_strategy":"primary","db_duration_s":0.004376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546vm-test-run-nixbot-gitlab> gitlab # [  158.052578] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42af9d9ab5efbe22ec21e3cb","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42af9d9ab5efbe22ec21e3cb: start","job_status":"start","queue_duration_s":5.274747,"scheduling_latency_s":5.274813}1547vm-test-run-nixbot-gitlab> gitlab # [  158.130796] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42af9d9ab5efbe22ec21e3cb","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42af9d9ab5efbe22ec21e3cb: done: 0.078389 sec","job_status":"done","queue_duration_s":5.274747,"scheduling_latency_s":5.274813,"redis_calls":10,"redis_duration_s":0.006625,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003692,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.026,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.026,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.078389,"completed_at":"2026-07-04T13:32:13.142Z","load_balancing_strategy":"primary","db_duration_s":0.006597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548vm-test-run-nixbot-gitlab> gitlab # [  158.195221] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d05439c5781c7b12d11398e6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d05439c5781c7b12d11398e6: start","job_status":"start","queue_duration_s":5.417593,"scheduling_latency_s":5.41766}1549vm-test-run-nixbot-gitlab> gitlab # [  158.253244] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d05439c5781c7b12d11398e6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d05439c5781c7b12d11398e6: done: 0.057908 sec","job_status":"done","queue_duration_s":5.417593,"scheduling_latency_s":5.41766,"redis_calls":10,"redis_duration_s":0.00575,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003666,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057908,"completed_at":"2026-07-04T13:32:13.264Z","load_balancing_strategy":"primary","db_duration_s":0.003155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [  158.317572] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e009eba66af9b236085b089c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e009eba66af9b236085b089c: start","job_status":"start","queue_duration_s":5.539681,"scheduling_latency_s":5.539745}1551vm-test-run-nixbot-gitlab> gitlab # [  158.373834] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e009eba66af9b236085b089c","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e009eba66af9b236085b089c: done: 0.056397 sec","job_status":"done","queue_duration_s":5.539681,"scheduling_latency_s":5.539745,"redis_calls":10,"redis_duration_s":0.0038850000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002197,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056397,"completed_at":"2026-07-04T13:32:13.385Z","load_balancing_strategy":"primary","db_duration_s":0.003225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552vm-test-run-nixbot-gitlab> gitlab # [  158.437629] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b544df3e66e891cbca455c0b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b544df3e66e891cbca455c0b: start","job_status":"start","queue_duration_s":5.660214,"scheduling_latency_s":5.660254}1553vm-test-run-nixbot-gitlab> gitlab # [  158.505446] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b544df3e66e891cbca455c0b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b544df3e66e891cbca455c0b: done: 0.067331 sec","job_status":"done","queue_duration_s":5.660214,"scheduling_latency_s":5.660254,"redis_calls":10,"redis_duration_s":0.007306,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004739,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067331,"completed_at":"2026-07-04T13:32:13.516Z","load_balancing_strategy":"primary","db_duration_s":0.003172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554vm-test-run-nixbot-gitlab> gitlab # [  158.568985] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8480104d63ad1277426f8a46","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8480104d63ad1277426f8a46: start","job_status":"start","queue_duration_s":5.791238,"scheduling_latency_s":5.791308}1555vm-test-run-nixbot-gitlab> gitlab # [  158.614963] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8480104d63ad1277426f8a46","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8480104d63ad1277426f8a46: done: 0.045995 sec","job_status":"done","queue_duration_s":5.791238,"scheduling_latency_s":5.791308,"redis_calls":10,"redis_duration_s":0.006317,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00275,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003567,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045995,"completed_at":"2026-07-04T13:32:13.626Z","load_balancing_strategy":"primary","db_duration_s":0.003892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556vm-test-run-nixbot-gitlab> gitlab # [  158.678843] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"abe40394e85bc541bbf69a4d","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-abe40394e85bc541bbf69a4d: start","job_status":"start","queue_duration_s":5.901004,"scheduling_latency_s":5.901093}1557vm-test-run-nixbot-gitlab> gitlab # [  158.722261] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"abe40394e85bc541bbf69a4d","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-abe40394e85bc541bbf69a4d: done: 0.043522 sec","job_status":"done","queue_duration_s":5.901004,"scheduling_latency_s":5.901093,"redis_calls":10,"redis_duration_s":0.006725,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004172,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043522,"completed_at":"2026-07-04T13:32:13.733Z","load_balancing_strategy":"primary","db_duration_s":0.003161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558vm-test-run-nixbot-gitlab> gitlab # [  158.785800] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7513c6fed77625256a487248","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7513c6fed77625256a487248: start","job_status":"start","queue_duration_s":6.008099,"scheduling_latency_s":6.008182}1559vm-test-run-nixbot-gitlab> gitlab # [  158.849464] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7513c6fed77625256a487248","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7513c6fed77625256a487248: done: 0.06349 sec","job_status":"done","queue_duration_s":6.008099,"scheduling_latency_s":6.008182,"redis_calls":10,"redis_duration_s":0.006638,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002438,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0042,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.06349,"completed_at":"2026-07-04T13:32:13.860Z","load_balancing_strategy":"primary","db_duration_s":0.003286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [  158.911868] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c29a2de68ff74f9ed4d218a3","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c29a2de68ff74f9ed4d218a3: start","job_status":"start","queue_duration_s":6.134029,"scheduling_latency_s":6.134114}1561vm-test-run-nixbot-gitlab> gitlab # [  158.955984] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c29a2de68ff74f9ed4d218a3","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c29a2de68ff74f9ed4d218a3: done: 0.043463 sec","job_status":"done","queue_duration_s":6.134029,"scheduling_latency_s":6.134114,"redis_calls":10,"redis_duration_s":0.0063360000000000005,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003976,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043463,"completed_at":"2026-07-04T13:32:13.966Z","load_balancing_strategy":"primary","db_duration_s":0.003174,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [  158.961177] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:13.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4dc7ccb75e8125b676093ab6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4dc7ccb75e8125b676093ab6: start","job_status":"start","queue_duration_s":6.183612,"scheduling_latency_s":6.183662}1563vm-test-run-nixbot-gitlab> gitlab # [  158.994756] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4dc7ccb75e8125b676093ab6","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4dc7ccb75e8125b676093ab6: done: 0.03389 sec","job_status":"done","queue_duration_s":6.183612,"scheduling_latency_s":6.183662,"redis_calls":10,"redis_duration_s":0.005116,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003333,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03389,"completed_at":"2026-07-04T13:32:14.006Z","load_balancing_strategy":"primary","db_duration_s":0.003753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [  159.027539] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5c69e97667a5c5453da1e04b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5c69e97667a5c5453da1e04b: start","job_status":"start","queue_duration_s":6.222694,"scheduling_latency_s":6.222716}1565vm-test-run-nixbot-gitlab> gitlab # [  159.047071] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5c69e97667a5c5453da1e04b","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5c69e97667a5c5453da1e04b: done: 0.02556 sec","job_status":"done","queue_duration_s":6.222694,"scheduling_latency_s":6.222716,"redis_calls":10,"redis_duration_s":0.011016,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003036,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02556,"completed_at":"2026-07-04T13:32:14.037Z","load_balancing_strategy":"primary","db_duration_s":0.001702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566vm-test-run-nixbot-gitlab> gitlab # [  159.147235] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"de6d8b45288b3be3efe44c88","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-de6d8b45288b3be3efe44c88: start","job_status":"start","queue_duration_s":6.250679,"scheduling_latency_s":6.2507}1567vm-test-run-nixbot-gitlab> gitlab # [  159.148407] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"de6d8b45288b3be3efe44c88","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-de6d8b45288b3be3efe44c88: done: 0.02086 sec","job_status":"done","queue_duration_s":6.250679,"scheduling_latency_s":6.2507,"redis_calls":10,"redis_duration_s":0.0072239999999999995,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003278,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003946,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02086,"completed_at":"2026-07-04T13:32:14.060Z","load_balancing_strategy":"primary","db_duration_s":0.001309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568vm-test-run-nixbot-gitlab> gitlab # [  159.216953] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5006233dbdf66748719b576","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5006233dbdf66748719b576: start","job_status":"start","queue_duration_s":6.27287,"scheduling_latency_s":6.27289}1569vm-test-run-nixbot-gitlab> gitlab # [  159.218247] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5006233dbdf66748719b576","created_at":"2026-07-04T13:32:07.703Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-3ffd6a48210e4cd1","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ba09b72132f80c518ae13fe2079ac255","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:07.789Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5006233dbdf66748719b576: done: 0.015751 sec","job_status":"done","queue_duration_s":6.27287,"scheduling_latency_s":6.27289,"redis_calls":10,"redis_duration_s":0.002358,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00147,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015751,"completed_at":"2026-07-04T13:32:14.077Z","load_balancing_strategy":"primary","db_duration_s":0.001381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570vm-test-run-nixbot-gitlab> gitlab # [  159.312364] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7746b171d73bb4a606858df4","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7746b171d73bb4a606858df4: start","job_status":"start","queue_duration_s":6.040551,"scheduling_latency_s":6.040573}1571vm-test-run-nixbot-gitlab> gitlab # [  159.314372] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7746b171d73bb4a606858df4","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7746b171d73bb4a606858df4: done: 0.022742 sec","job_status":"done","queue_duration_s":6.040551,"scheduling_latency_s":6.040573,"redis_calls":16,"redis_duration_s":0.003645,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001987,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384127,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022742,"completed_at":"2026-07-04T13:32:14.101Z","load_balancing_strategy":"primary","db_duration_s":0.003052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572vm-test-run-nixbot-gitlab> gitlab # [  159.352754] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a2aba427558490e6ea5643e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a2aba427558490e6ea5643e: start","job_status":"start","queue_duration_s":6.064605,"scheduling_latency_s":6.064625}1573vm-test-run-nixbot-gitlab> gitlab # [  159.430424] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a2aba427558490e6ea5643e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a2aba427558490e6ea5643e: done: 0.014977 sec","job_status":"done","queue_duration_s":6.064605,"scheduling_latency_s":6.064625,"redis_calls":13,"redis_duration_s":0.002891,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014977,"completed_at":"2026-07-04T13:32:14.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574vm-test-run-nixbot-gitlab> gitlab # [  159.433074] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d89e4785eeb08bad057631","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d89e4785eeb08bad057631: start","job_status":"start","queue_duration_s":6.080858,"scheduling_latency_s":6.080879}1575vm-test-run-nixbot-gitlab> gitlab # [  159.453120] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d89e4785eeb08bad057631","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d89e4785eeb08bad057631: done: 0.014688 sec","job_status":"done","queue_duration_s":6.080858,"scheduling_latency_s":6.080879,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014688,"completed_at":"2026-07-04T13:32:14.134Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576vm-test-run-nixbot-gitlab> gitlab # [  159.532275] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00e0df4e012bc5f6a64c09e6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00e0df4e012bc5f6a64c09e6: start","job_status":"start","queue_duration_s":6.09683,"scheduling_latency_s":6.096851}1577vm-test-run-nixbot-gitlab> gitlab # [  159.599517] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00e0df4e012bc5f6a64c09e6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00e0df4e012bc5f6a64c09e6: done: 0.014703 sec","job_status":"done","queue_duration_s":6.09683,"scheduling_latency_s":6.096851,"redis_calls":13,"redis_duration_s":0.002845,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014703,"completed_at":"2026-07-04T13:32:14.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578vm-test-run-nixbot-gitlab> gitlab # [  159.601166] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"722e2864b0eb2fc3005f4e09","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-722e2864b0eb2fc3005f4e09: start","job_status":"start","queue_duration_s":6.112735,"scheduling_latency_s":6.112789}1579vm-test-run-nixbot-gitlab> gitlab # [  159.670534] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"722e2864b0eb2fc3005f4e09","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-722e2864b0eb2fc3005f4e09: done: 0.017987 sec","job_status":"done","queue_duration_s":6.112735,"scheduling_latency_s":6.112789,"redis_calls":13,"redis_duration_s":0.003616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017987,"completed_at":"2026-07-04T13:32:14.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580vm-test-run-nixbot-gitlab> gitlab # [  159.673578] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a3b540a0a4b6f34054ea504","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a3b540a0a4b6f34054ea504: start","job_status":"start","queue_duration_s":6.132316,"scheduling_latency_s":6.132343}1581vm-test-run-nixbot-gitlab> gitlab # [  159.751962] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a3b540a0a4b6f34054ea504","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a3b540a0a4b6f34054ea504: done: 0.015808 sec","job_status":"done","queue_duration_s":6.132316,"scheduling_latency_s":6.132343,"redis_calls":13,"redis_duration_s":0.002951,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015808,"completed_at":"2026-07-04T13:32:14.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582vm-test-run-nixbot-gitlab> gitlab # [  159.754351] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fed8030bcb8ab72799f44c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fed8030bcb8ab72799f44c: start","job_status":"start","queue_duration_s":6.149371,"scheduling_latency_s":6.149392}1583vm-test-run-nixbot-gitlab> gitlab # [  159.774375] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fed8030bcb8ab72799f44c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fed8030bcb8ab72799f44c: done: 0.015124 sec","job_status":"done","queue_duration_s":6.149371,"scheduling_latency_s":6.149392,"redis_calls":13,"redis_duration_s":0.002839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015124,"completed_at":"2026-07-04T13:32:14.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [  159.859164] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f55bd61763b2633f372f57","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f55bd61763b2633f372f57: start","job_status":"start","queue_duration_s":6.165796,"scheduling_latency_s":6.165817}1585vm-test-run-nixbot-gitlab> gitlab # [  159.860277] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f55bd61763b2633f372f57","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f55bd61763b2633f372f57: done: 0.014833 sec","job_status":"done","queue_duration_s":6.165796,"scheduling_latency_s":6.165817,"redis_calls":13,"redis_duration_s":0.00292,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014833,"completed_at":"2026-07-04T13:32:14.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [  159.961504] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ed2007a2b66b565b3476ea","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ed2007a2b66b565b3476ea: start","job_status":"start","queue_duration_s":6.181898,"scheduling_latency_s":6.181918}1587vm-test-run-nixbot-gitlab> gitlab # [  159.962757] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5ed2007a2b66b565b3476ea","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5ed2007a2b66b565b3476ea: done: 0.019124 sec","job_status":"done","queue_duration_s":6.181898,"scheduling_latency_s":6.181918,"redis_calls":13,"redis_duration_s":0.007233000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005708,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019124,"completed_at":"2026-07-04T13:32:14.239Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [  159.998178] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d720005490a4a3ead5835d78","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d720005490a4a3ead5835d78: start","job_status":"start","queue_duration_s":6.202417,"scheduling_latency_s":6.202438}1589vm-test-run-nixbot-gitlab> gitlab # [  160.067942] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d720005490a4a3ead5835d78","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d720005490a4a3ead5835d78: done: 0.015343 sec","job_status":"done","queue_duration_s":6.202417,"scheduling_latency_s":6.202438,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015343,"completed_at":"2026-07-04T13:32:14.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590vm-test-run-nixbot-gitlab> gitlab # [  160.078327] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6067c902287ace1c9d60d804","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6067c902287ace1c9d60d804: start","job_status":"start","queue_duration_s":6.21906,"scheduling_latency_s":6.219081}1591vm-test-run-nixbot-gitlab> gitlab # [  160.098357] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6067c902287ace1c9d60d804","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6067c902287ace1c9d60d804: done: 0.015411 sec","job_status":"done","queue_duration_s":6.21906,"scheduling_latency_s":6.219081,"redis_calls":13,"redis_duration_s":0.003195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015411,"completed_at":"2026-07-04T13:32:14.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592vm-test-run-nixbot-gitlab> gitlab # [  160.169996] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba537b08537c0fe0d835a7e8","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba537b08537c0fe0d835a7e8: start","job_status":"start","queue_duration_s":6.23582,"scheduling_latency_s":6.23584}1593vm-test-run-nixbot-gitlab> gitlab # [  160.179162] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba537b08537c0fe0d835a7e8","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba537b08537c0fe0d835a7e8: done: 0.015171 sec","job_status":"done","queue_duration_s":6.23582,"scheduling_latency_s":6.23584,"redis_calls":13,"redis_duration_s":0.002972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015171,"completed_at":"2026-07-04T13:32:14.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594vm-test-run-nixbot-gitlab> gitlab # [  160.242093] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e1a7cb56502f38076d24d91","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e1a7cb56502f38076d24d91: start","job_status":"start","queue_duration_s":6.252267,"scheduling_latency_s":6.252288}1595vm-test-run-nixbot-gitlab> gitlab # [  160.313477] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e1a7cb56502f38076d24d91","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e1a7cb56502f38076d24d91: done: 0.015186 sec","job_status":"done","queue_duration_s":6.252267,"scheduling_latency_s":6.252288,"redis_calls":13,"redis_duration_s":0.003156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015186,"completed_at":"2026-07-04T13:32:14.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596vm-test-run-nixbot-gitlab> gitlab # [  160.322405] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7f9dcdc9978e5119e8cb02a","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7f9dcdc9978e5119e8cb02a: start","job_status":"start","queue_duration_s":6.268827,"scheduling_latency_s":6.268848}1597vm-test-run-nixbot-gitlab> gitlab # [  160.342911] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7f9dcdc9978e5119e8cb02a","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7f9dcdc9978e5119e8cb02a: done: 0.015236 sec","job_status":"done","queue_duration_s":6.268827,"scheduling_latency_s":6.268848,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015236,"completed_at":"2026-07-04T13:32:14.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1598vm-test-run-nixbot-gitlab> gitlab # [  160.445136] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0f3a8c53a5403bd1f95bbe8","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0f3a8c53a5403bd1f95bbe8: start","job_status":"start","queue_duration_s":6.285345,"scheduling_latency_s":6.285366}1599vm-test-run-nixbot-gitlab> gitlab # [  160.446241] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0f3a8c53a5403bd1f95bbe8","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0f3a8c53a5403bd1f95bbe8: done: 0.015582 sec","job_status":"done","queue_duration_s":6.285345,"scheduling_latency_s":6.285366,"redis_calls":13,"redis_duration_s":0.003155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015582,"completed_at":"2026-07-04T13:32:14.339Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1600vm-test-run-nixbot-gitlab> gitlab # [  160.486090] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"761a6fc62952aa337bdb6daa","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-761a6fc62952aa337bdb6daa: start","job_status":"start","queue_duration_s":6.302226,"scheduling_latency_s":6.302246}1601vm-test-run-nixbot-gitlab> gitlab # [  160.506121] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"761a6fc62952aa337bdb6daa","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-761a6fc62952aa337bdb6daa: done: 0.015758 sec","job_status":"done","queue_duration_s":6.302226,"scheduling_latency_s":6.302246,"redis_calls":13,"redis_duration_s":0.003036,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015758,"completed_at":"2026-07-04T13:32:14.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602vm-test-run-nixbot-gitlab> gitlab # [  160.594177] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:14.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b02752fc6e2df67b922eb67","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b02752fc6e2df67b922eb67: start","job_status":"start","queue_duration_s":6.319311,"scheduling_latency_s":6.319332}1603vm-test-run-nixbot-gitlab> gitlab # [  160.595707] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b02752fc6e2df67b922eb67","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b02752fc6e2df67b922eb67: done: 1.206583 sec","job_status":"done","queue_duration_s":6.319311,"scheduling_latency_s":6.319332,"redis_calls":14,"redis_duration_s":1.191932,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":1.188477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":1.206583,"completed_at":"2026-07-04T13:32:15.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1604vm-test-run-nixbot-gitlab> gitlab # [  160.650161] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07d456a491b6c8a3ee9bddbe","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07d456a491b6c8a3ee9bddbe: start","job_status":"start","queue_duration_s":7.527321,"scheduling_latency_s":7.527348}1605vm-test-run-nixbot-gitlab> gitlab # [  160.670135] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07d456a491b6c8a3ee9bddbe","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07d456a491b6c8a3ee9bddbe: done: 0.015334 sec","job_status":"done","queue_duration_s":7.527321,"scheduling_latency_s":7.527348,"redis_calls":13,"redis_duration_s":0.002844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015334,"completed_at":"2026-07-04T13:32:15.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1606vm-test-run-nixbot-gitlab> gitlab # [  160.747147] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb41f1c3e053c91d0314106","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb41f1c3e053c91d0314106: start","job_status":"start","queue_duration_s":7.543914,"scheduling_latency_s":7.543935}1607vm-test-run-nixbot-gitlab> gitlab # [  160.751201] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdb41f1c3e053c91d0314106","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdb41f1c3e053c91d0314106: done: 0.014656 sec","job_status":"done","queue_duration_s":7.543914,"scheduling_latency_s":7.543935,"redis_calls":13,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014656,"completed_at":"2026-07-04T13:32:15.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608vm-test-run-nixbot-gitlab> gitlab # [  160.814263] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41e5d3e8239ea43da0eae6f1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41e5d3e8239ea43da0eae6f1: start","job_status":"start","queue_duration_s":7.559857,"scheduling_latency_s":7.559879}1609vm-test-run-nixbot-gitlab> gitlab # [  160.890931] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41e5d3e8239ea43da0eae6f1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41e5d3e8239ea43da0eae6f1: done: 0.019895 sec","job_status":"done","queue_duration_s":7.559857,"scheduling_latency_s":7.559879,"redis_calls":13,"redis_duration_s":0.008015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00598,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019895,"completed_at":"2026-07-04T13:32:15.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1610vm-test-run-nixbot-gitlab> gitlab # [  160.895228] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ae920c2a9afda41d97aabe3","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ae920c2a9afda41d97aabe3: start","job_status":"start","queue_duration_s":7.583071,"scheduling_latency_s":7.583093}1611vm-test-run-nixbot-gitlab> gitlab # [  160.915284] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ae920c2a9afda41d97aabe3","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ae920c2a9afda41d97aabe3: done: 0.022658 sec","job_status":"done","queue_duration_s":7.583071,"scheduling_latency_s":7.583093,"redis_calls":13,"redis_duration_s":0.01031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003385,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022658,"completed_at":"2026-07-04T13:32:15.644Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1612vm-test-run-nixbot-gitlab> gitlab # [  161.035565] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bf95ca0f9a7a7b2313f215e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bf95ca0f9a7a7b2313f215e: start","job_status":"start","queue_duration_s":7.609073,"scheduling_latency_s":7.609094}1613vm-test-run-nixbot-gitlab> gitlab # [  161.036847] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bf95ca0f9a7a7b2313f215e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bf95ca0f9a7a7b2313f215e: done: 0.020351 sec","job_status":"done","queue_duration_s":7.609073,"scheduling_latency_s":7.609094,"redis_calls":13,"redis_duration_s":0.007929,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002296,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005633,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020351,"completed_at":"2026-07-04T13:32:15.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1614vm-test-run-nixbot-gitlab> gitlab # [  161.059073] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f6ecde98e5a15f2e38bb4aa","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f6ecde98e5a15f2e38bb4aa: start","job_status":"start","queue_duration_s":7.630716,"scheduling_latency_s":7.630738}1615vm-test-run-nixbot-gitlab> gitlab # [  161.078795] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f6ecde98e5a15f2e38bb4aa","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f6ecde98e5a15f2e38bb4aa: done: 0.015177 sec","job_status":"done","queue_duration_s":7.630716,"scheduling_latency_s":7.630738,"redis_calls":13,"redis_duration_s":0.003104,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015177,"completed_at":"2026-07-04T13:32:15.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1616vm-test-run-nixbot-gitlab> gitlab # [  161.160289] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a01da419265f798a6a8283c1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a01da419265f798a6a8283c1: start","job_status":"start","queue_duration_s":7.647157,"scheduling_latency_s":7.647178}1617vm-test-run-nixbot-gitlab> gitlab # [  161.161488] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a01da419265f798a6a8283c1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a01da419265f798a6a8283c1: done: 0.01507 sec","job_status":"done","queue_duration_s":7.647157,"scheduling_latency_s":7.647178,"redis_calls":13,"redis_duration_s":0.002986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01507,"completed_at":"2026-07-04T13:32:15.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1618vm-test-run-nixbot-gitlab> gitlab # [  161.287964] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dc2111706fe90bdbf83039e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dc2111706fe90bdbf83039e: start","job_status":"start","queue_duration_s":7.663506,"scheduling_latency_s":7.663527}1619vm-test-run-nixbot-gitlab> gitlab # [  161.289116] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dc2111706fe90bdbf83039e","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dc2111706fe90bdbf83039e: done: 0.014963 sec","job_status":"done","queue_duration_s":7.663506,"scheduling_latency_s":7.663527,"redis_calls":13,"redis_duration_s":0.002964,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014963,"completed_at":"2026-07-04T13:32:15.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1620vm-test-run-nixbot-gitlab> gitlab # [  161.302436] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3206ec2ee93157817a519396","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3206ec2ee93157817a519396: start","job_status":"start","queue_duration_s":7.679904,"scheduling_latency_s":7.679925}1621vm-test-run-nixbot-gitlab> gitlab # [  161.376066] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3206ec2ee93157817a519396","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3206ec2ee93157817a519396: done: 0.014837 sec","job_status":"done","queue_duration_s":7.679904,"scheduling_latency_s":7.679925,"redis_calls":13,"redis_duration_s":0.003013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014837,"completed_at":"2026-07-04T13:32:15.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1622vm-test-run-nixbot-gitlab> gitlab # [  161.382262] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"064940c945be9b5c57fc3f26","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-064940c945be9b5c57fc3f26: start","job_status":"start","queue_duration_s":7.696068,"scheduling_latency_s":7.696089}1623vm-test-run-nixbot-gitlab> gitlab # [  161.452315] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"064940c945be9b5c57fc3f26","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-064940c945be9b5c57fc3f26: done: 0.014844 sec","job_status":"done","queue_duration_s":7.696068,"scheduling_latency_s":7.696089,"redis_calls":13,"redis_duration_s":0.002965,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014844,"completed_at":"2026-07-04T13:32:15.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1624vm-test-run-nixbot-gitlab> gitlab # [  161.521431] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5059a23e9030e9d1de8643","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5059a23e9030e9d1de8643: start","job_status":"start","queue_duration_s":7.712207,"scheduling_latency_s":7.712227}1625vm-test-run-nixbot-gitlab> gitlab # [  161.522712] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b5059a23e9030e9d1de8643","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b5059a23e9030e9d1de8643: done: 0.015264 sec","job_status":"done","queue_duration_s":7.712207,"scheduling_latency_s":7.712227,"redis_calls":13,"redis_duration_s":0.00326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015264,"completed_at":"2026-07-04T13:32:15.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1626vm-test-run-nixbot-gitlab> gitlab # [  161.600207] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eac8aaf56b4d8d95d969469","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eac8aaf56b4d8d95d969469: start","job_status":"start","queue_duration_s":7.728856,"scheduling_latency_s":7.728876}1627vm-test-run-nixbot-gitlab> gitlab # [  161.601271] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eac8aaf56b4d8d95d969469","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eac8aaf56b4d8d95d969469: done: 0.014996 sec","job_status":"done","queue_duration_s":7.728856,"scheduling_latency_s":7.728876,"redis_calls":13,"redis_duration_s":0.0029309999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014996,"completed_at":"2026-07-04T13:32:15.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1628vm-test-run-nixbot-gitlab> gitlab # [  161.671191] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53a8641b6487cc16e5fdaf47","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53a8641b6487cc16e5fdaf47: start","job_status":"start","queue_duration_s":7.745143,"scheduling_latency_s":7.745163}1629vm-test-run-nixbot-gitlab> gitlab # [  161.672747] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53a8641b6487cc16e5fdaf47","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53a8641b6487cc16e5fdaf47: done: 0.014886 sec","job_status":"done","queue_duration_s":7.745143,"scheduling_latency_s":7.745163,"redis_calls":13,"redis_duration_s":0.003104,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014886,"completed_at":"2026-07-04T13:32:15.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1630vm-test-run-nixbot-gitlab> gitlab # [  161.740075] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f159c97c3339d9778b0ff39c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f159c97c3339d9778b0ff39c: start","job_status":"start","queue_duration_s":7.761393,"scheduling_latency_s":7.761414}1631vm-test-run-nixbot-gitlab> gitlab # [  161.741256] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f159c97c3339d9778b0ff39c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f159c97c3339d9778b0ff39c: done: 0.014998 sec","job_status":"done","queue_duration_s":7.761393,"scheduling_latency_s":7.761414,"redis_calls":13,"redis_duration_s":0.003086,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014998,"completed_at":"2026-07-04T13:32:15.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1632vm-test-run-nixbot-gitlab> gitlab # [  161.835385] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2e6d5a870a5ff63fca08069","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2e6d5a870a5ff63fca08069: start","job_status":"start","queue_duration_s":7.777664,"scheduling_latency_s":7.777685}1633vm-test-run-nixbot-gitlab> gitlab # [  161.837178] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2e6d5a870a5ff63fca08069","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2e6d5a870a5ff63fca08069: done: 0.014894 sec","job_status":"done","queue_duration_s":7.777664,"scheduling_latency_s":7.777685,"redis_calls":13,"redis_duration_s":0.0029579999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014894,"completed_at":"2026-07-04T13:32:15.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1634vm-test-run-nixbot-gitlab> gitlab # [  161.907221] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb09ea2e4f7e753ad7e4aa2c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb09ea2e4f7e753ad7e4aa2c: start","job_status":"start","queue_duration_s":7.793893,"scheduling_latency_s":7.793913}1635vm-test-run-nixbot-gitlab> gitlab # [  161.908341] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb09ea2e4f7e753ad7e4aa2c","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb09ea2e4f7e753ad7e4aa2c: done: 0.017156 sec","job_status":"done","queue_duration_s":7.793893,"scheduling_latency_s":7.793913,"redis_calls":13,"redis_duration_s":0.0032660000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017156,"completed_at":"2026-07-04T13:32:15.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1636vm-test-run-nixbot-gitlab> gitlab # [  161.978337] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c1f27f28da9112cfbb13afe","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c1f27f28da9112cfbb13afe: start","job_status":"start","queue_duration_s":7.812614,"scheduling_latency_s":7.81264}1637vm-test-run-nixbot-gitlab> gitlab # [  161.979420] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c1f27f28da9112cfbb13afe","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c1f27f28da9112cfbb13afe: done: 0.01575 sec","job_status":"done","queue_duration_s":7.812614,"scheduling_latency_s":7.81264,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01575,"completed_at":"2026-07-04T13:32:15.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1638vm-test-run-nixbot-gitlab> gitlab # [  162.068437] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d5d6934bfc201c5549f79b1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d5d6934bfc201c5549f79b1: start","job_status":"start","queue_duration_s":7.829585,"scheduling_latency_s":7.829605}1639vm-test-run-nixbot-gitlab> gitlab # [  162.070100] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d5d6934bfc201c5549f79b1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d5d6934bfc201c5549f79b1: done: 0.014651 sec","job_status":"done","queue_duration_s":7.829585,"scheduling_latency_s":7.829605,"redis_calls":13,"redis_duration_s":0.002727,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014651,"completed_at":"2026-07-04T13:32:15.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1640vm-test-run-nixbot-gitlab> gitlab # [  162.138260] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16849e11e48b3fe1d4e871b7","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16849e11e48b3fe1d4e871b7: start","job_status":"start","queue_duration_s":7.845469,"scheduling_latency_s":7.845489}1641vm-test-run-nixbot-gitlab> gitlab # [  162.139800] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16849e11e48b3fe1d4e871b7","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16849e11e48b3fe1d4e871b7: done: 0.014681 sec","job_status":"done","queue_duration_s":7.845469,"scheduling_latency_s":7.845489,"redis_calls":13,"redis_duration_s":0.002845,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-07-04T13:32:15.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1642vm-test-run-nixbot-gitlab> gitlab # [  162.216450] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cdf58d5aa18946becc2870d","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cdf58d5aa18946becc2870d: start","job_status":"start","queue_duration_s":7.861406,"scheduling_latency_s":7.861427}1643vm-test-run-nixbot-gitlab> gitlab # [  162.218102] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cdf58d5aa18946becc2870d","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cdf58d5aa18946becc2870d: done: 0.015567 sec","job_status":"done","queue_duration_s":7.861406,"scheduling_latency_s":7.861427,"redis_calls":13,"redis_duration_s":0.0036750000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002768,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015567,"completed_at":"2026-07-04T13:32:15.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1644vm-test-run-nixbot-gitlab> gitlab # [  162.296269] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c145ad240a62f3a22382833","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c145ad240a62f3a22382833: start","job_status":"start","queue_duration_s":7.878381,"scheduling_latency_s":7.878402}1645vm-test-run-nixbot-gitlab> gitlab # [  162.297371] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c145ad240a62f3a22382833","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c145ad240a62f3a22382833: done: 0.015042 sec","job_status":"done","queue_duration_s":7.878381,"scheduling_latency_s":7.878402,"redis_calls":13,"redis_duration_s":0.00306,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015042,"completed_at":"2026-07-04T13:32:15.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1646vm-test-run-nixbot-gitlab> gitlab # [  162.378062] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be50a0bcbd2c22fdc201be54","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be50a0bcbd2c22fdc201be54: start","job_status":"start","queue_duration_s":7.894804,"scheduling_latency_s":7.894825}1647vm-test-run-nixbot-gitlab> gitlab # [  162.379501] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be50a0bcbd2c22fdc201be54","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be50a0bcbd2c22fdc201be54: done: 0.014966 sec","job_status":"done","queue_duration_s":7.894804,"scheduling_latency_s":7.894825,"redis_calls":13,"redis_duration_s":0.0029960000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014966,"completed_at":"2026-07-04T13:32:15.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1648vm-test-run-nixbot-gitlab> gitlab # [  162.460140] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d50adec81674c6b54137e67","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d50adec81674c6b54137e67: start","job_status":"start","queue_duration_s":7.911103,"scheduling_latency_s":7.911124}1649vm-test-run-nixbot-gitlab> gitlab # [  162.461282] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d50adec81674c6b54137e67","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d50adec81674c6b54137e67: done: 0.015305 sec","job_status":"done","queue_duration_s":7.911103,"scheduling_latency_s":7.911124,"redis_calls":13,"redis_duration_s":0.0032500000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015305,"completed_at":"2026-07-04T13:32:15.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1650vm-test-run-nixbot-gitlab> gitlab # [  162.491246] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbab11f067c816fbfb40faed","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbab11f067c816fbfb40faed: start","job_status":"start","queue_duration_s":7.927703,"scheduling_latency_s":7.927724}1651vm-test-run-nixbot-gitlab> gitlab # [  162.561186] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbab11f067c816fbfb40faed","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbab11f067c816fbfb40faed: done: 0.014984 sec","job_status":"done","queue_duration_s":7.927703,"scheduling_latency_s":7.927724,"redis_calls":13,"redis_duration_s":0.003069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014984,"completed_at":"2026-07-04T13:32:15.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1652vm-test-run-nixbot-gitlab> gitlab # [  162.633324] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d10eef25df58e1dbc25579d6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d10eef25df58e1dbc25579d6: start","job_status":"start","queue_duration_s":7.944083,"scheduling_latency_s":7.944103}1653vm-test-run-nixbot-gitlab> gitlab # [  162.634521] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d10eef25df58e1dbc25579d6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d10eef25df58e1dbc25579d6: done: 0.014535 sec","job_status":"done","queue_duration_s":7.944083,"scheduling_latency_s":7.944103,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014535,"completed_at":"2026-07-04T13:32:15.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1654vm-test-run-nixbot-gitlab> gitlab # [  162.652090] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:15.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2819b6343c5513bcccc4c4b1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2819b6343c5513bcccc4c4b1: start","job_status":"start","queue_duration_s":7.959921,"scheduling_latency_s":7.959941}1655vm-test-run-nixbot-gitlab> gitlab # [  162.722540] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2819b6343c5513bcccc4c4b1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2819b6343c5513bcccc4c4b1: done: 0.014681 sec","job_status":"done","queue_duration_s":7.959921,"scheduling_latency_s":7.959941,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-07-04T13:32:16.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1656vm-test-run-nixbot-gitlab> gitlab # [  162.732043] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ce9203d228b11855a12a0f2","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ce9203d228b11855a12a0f2: start","job_status":"start","queue_duration_s":7.975903,"scheduling_latency_s":7.975924}1657vm-test-run-nixbot-gitlab> gitlab # [  162.805111] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ce9203d228b11855a12a0f2","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ce9203d228b11855a12a0f2: done: 0.01463 sec","job_status":"done","queue_duration_s":7.975903,"scheduling_latency_s":7.975924,"redis_calls":13,"redis_duration_s":0.002842,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01463,"completed_at":"2026-07-04T13:32:16.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1658vm-test-run-nixbot-gitlab> gitlab # [  162.812363] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab2f17cf686fff27edb2df5","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab2f17cf686fff27edb2df5: start","job_status":"start","queue_duration_s":7.991851,"scheduling_latency_s":7.991871}1659vm-test-run-nixbot-gitlab> gitlab # [  162.897173] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ab2f17cf686fff27edb2df5","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ab2f17cf686fff27edb2df5: done: 0.014602 sec","job_status":"done","queue_duration_s":7.991851,"scheduling_latency_s":7.991871,"redis_calls":13,"redis_duration_s":0.0028049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014602,"completed_at":"2026-07-04T13:32:16.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1660vm-test-run-nixbot-gitlab> gitlab # [  162.899192] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6e5367c37a6452b8d576d1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6e5367c37a6452b8d576d1: start","job_status":"start","queue_duration_s":8.0077,"scheduling_latency_s":8.007721}1661vm-test-run-nixbot-gitlab> gitlab # [  162.913223] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c6e5367c37a6452b8d576d1","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c6e5367c37a6452b8d576d1: done: 0.018314 sec","job_status":"done","queue_duration_s":8.0077,"scheduling_latency_s":8.007721,"redis_calls":13,"redis_duration_s":0.003044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018314,"completed_at":"2026-07-04T13:32:16.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1662vm-test-run-nixbot-gitlab> gitlab # [  162.999364] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:16.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5c2d7810f2bd970eef207b","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5c2d7810f2bd970eef207b: start","job_status":"start","queue_duration_s":8.027292,"scheduling_latency_s":8.027312}1663vm-test-run-nixbot-gitlab> gitlab # [  163.076581] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5c2d7810f2bd970eef207b","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5c2d7810f2bd970eef207b: done: 2.022221 sec","job_status":"done","queue_duration_s":8.027292,"scheduling_latency_s":8.027312,"redis_calls":13,"redis_duration_s":0.0031479999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.022221,"completed_at":"2026-07-04T13:32:18.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1664vm-test-run-nixbot-gitlab> gitlab # [  163.139178] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3024e03341115e6361727cb","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3024e03341115e6361727cb: start","job_status":"start","queue_duration_s":10.112468,"scheduling_latency_s":10.112506}1665vm-test-run-nixbot-gitlab> gitlab # [  163.157107] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3024e03341115e6361727cb","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3024e03341115e6361727cb: done: 0.018006 sec","job_status":"done","queue_duration_s":10.112468,"scheduling_latency_s":10.112506,"redis_calls":13,"redis_duration_s":0.003373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002419,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018006,"completed_at":"2026-07-04T13:32:18.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1666vm-test-run-nixbot-gitlab> gitlab # [  163.160434] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b17c7b52a7e7fcaa9ad7f200","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b17c7b52a7e7fcaa9ad7f200: start","job_status":"start","queue_duration_s":10.133938,"scheduling_latency_s":10.133959}1667vm-test-run-nixbot-gitlab> gitlab # [  163.179363] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b17c7b52a7e7fcaa9ad7f200","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b17c7b52a7e7fcaa9ad7f200: done: 0.018821 sec","job_status":"done","queue_duration_s":10.133938,"scheduling_latency_s":10.133959,"redis_calls":13,"redis_duration_s":0.00376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018821,"completed_at":"2026-07-04T13:32:18.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1668vm-test-run-nixbot-gitlab> gitlab # [  163.232805] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db83e8dc7eca2b543cdb9dd6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db83e8dc7eca2b543cdb9dd6: start","job_status":"start","queue_duration_s":10.159117,"scheduling_latency_s":10.15914}1669vm-test-run-nixbot-gitlab> gitlab # [  163.252345] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db83e8dc7eca2b543cdb9dd6","created_at":"2026-07-04T13:32:07.961Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-05f7c6c1d58943ee","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f9a17eef2c632da4b1d7302189b7771c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.038Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db83e8dc7eca2b543cdb9dd6: done: 0.024816 sec","job_status":"done","queue_duration_s":10.159117,"scheduling_latency_s":10.15914,"redis_calls":13,"redis_duration_s":0.011051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00362,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007431,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024816,"completed_at":"2026-07-04T13:32:18.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1670vm-test-run-nixbot-gitlab> gitlab # [  163.353153] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.226Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"56a89d27f6a6b4c005ab4a12","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-56a89d27f6a6b4c005ab4a12: start","job_status":"start","queue_duration_s":9.779476,"scheduling_latency_s":9.779498}1671vm-test-run-nixbot-gitlab> gitlab # [  163.354768] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.248Z","message":"No existing merge request to be cleaned up."}1672vm-test-run-nixbot-gitlab> gitlab # [  163.355832] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.259Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"56a89d27f6a6b4c005ab4a12","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-56a89d27f6a6b4c005ab4a12: done: 0.032793 sec","job_status":"done","queue_duration_s":9.779476,"scheduling_latency_s":9.779498,"redis_calls":13,"redis_duration_s":0.008679,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357028,"duration_s":0.032793,"completed_at":"2026-07-04T13:32:18.259Z","load_balancing_strategy":"primary","db_duration_s":0.005904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1673vm-test-run-nixbot-gitlab> gitlab # [  163.462816] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.260Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d87bfb3047084e5442a7120","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d87bfb3047084e5442a7120: start","job_status":"start","queue_duration_s":9.813601,"scheduling_latency_s":9.813621}1674vm-test-run-nixbot-gitlab> gitlab # [  163.464124] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.270Z","message":"No existing merge request to be cleaned up."}1675vm-test-run-nixbot-gitlab> gitlab # [  163.465104] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.277Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4d87bfb3047084e5442a7120","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4d87bfb3047084e5442a7120: done: 0.016254 sec","job_status":"done","queue_duration_s":9.813601,"scheduling_latency_s":9.813621,"redis_calls":10,"redis_duration_s":0.002464,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001487,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016254,"completed_at":"2026-07-04T13:32:18.277Z","load_balancing_strategy":"primary","db_duration_s":0.001345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1676vm-test-run-nixbot-gitlab> gitlab # [  163.542797] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.278Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04ef90fbf557f9498f68eb81","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04ef90fbf557f9498f68eb81: start","job_status":"start","queue_duration_s":9.831229,"scheduling_latency_s":9.83125}1677vm-test-run-nixbot-gitlab> gitlab # [  163.544082] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.288Z","message":"No existing merge request to be cleaned up."}1678vm-test-run-nixbot-gitlab> gitlab # [  163.544680] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.294Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"04ef90fbf557f9498f68eb81","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-04ef90fbf557f9498f68eb81: done: 0.015764 sec","job_status":"done","queue_duration_s":9.831229,"scheduling_latency_s":9.83125,"redis_calls":10,"redis_duration_s":0.002444,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015764,"completed_at":"2026-07-04T13:32:18.294Z","load_balancing_strategy":"primary","db_duration_s":0.001409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1679vm-test-run-nixbot-gitlab> gitlab # [  163.562320] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.296Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b9e030031a9b7bd129b3c6f5","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b9e030031a9b7bd129b3c6f5: start","job_status":"start","queue_duration_s":9.848798,"scheduling_latency_s":9.848818}1680vm-test-run-nixbot-gitlab> gitlab # [  163.634804] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.305Z","message":"No existing merge request to be cleaned up."}1681vm-test-run-nixbot-gitlab> gitlab # [  163.636197] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:18.311Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b9e030031a9b7bd129b3c6f5","created_at":"2026-07-04T13:32:08.438Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-5dcb87e076e643a7","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4569a4b3dd4064e86585540b74aab638","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:08.447Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b9e030031a9b7bd129b3c6f5: done: 0.015868 sec","job_status":"done","queue_duration_s":9.848798,"scheduling_latency_s":9.848818,"redis_calls":10,"redis_duration_s":0.002336,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015868,"completed_at":"2026-07-04T13:32:18.311Z","load_balancing_strategy":"primary","db_duration_s":0.001688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1682vm-test-run-nixbot-gitlab> gitlab # [  174.084725] setup-gitlab-start[2722]: W, [2026-07-04T13:32:29.095840 #2722]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1683vm-test-run-nixbot-gitlab> gitlab # [  196.725804] runuser[2721]: pam_unix(runuser:session): session closed for user gitlab1684vm-test-run-nixbot-gitlab> gitlab # [  196.730340] setup-gitlab-start[2953]: ++ jq -r .token1685vm-test-run-nixbot-gitlab> gitlab # [  196.732297] setup-gitlab-start[2954]: +++ date -d '+7 days' +%F1686vm-test-run-nixbot-gitlab> gitlab # [  196.735653] setup-gitlab-start[2952]: ++ curl -fs -X POST -H 'Authorization: Bearer 348aa883e9ec738a2e684e7d73b6de5f67a3b2238f70263520fcd29a2e1a4e28' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-11"}' http://gitlab/api/v4/users/1/personal_access_tokens1687vm-test-run-nixbot-gitlab> gitlab # [  197.200352] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:52.210Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9d241ea9ce50044db18d0e43","created_at":"2026-07-04T13:32:52.184Z","trace_propagation_headers":{"sentry-trace":"7b0eac1ceee846218476d54c3b673135-bc400282c6884625","baggage":"sentry-trace_id=7b0eac1ceee846218476d54c3b673135,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9YNNASMP0A4WXQMGDYMK","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9ed97a349c1bceda2326a8dad6d333d1d1ac76a447be34d4d32468d96a073f77","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:52.206Z","job_size_bytes":451,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9d241ea9ce50044db18d0e43: start","job_status":"start","queue_duration_s":0.004036,"scheduling_latency_s":0.004102}1688vm-test-run-nixbot-gitlab> gitlab # [  197.221305] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:52 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 4671689vm-test-run-nixbot-gitlab> gitlab # [  197.267051] setup-gitlab-start[2679]: + TOKEN=glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle1690vm-test-run-nixbot-gitlab> gitlab # [  197.267570] setup-gitlab-start[2679]: + test -n glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle1691vm-test-run-nixbot-gitlab> gitlab # [  197.268708] setup-gitlab-start[2679]: + test glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle '!=' null1692vm-test-run-nixbot-gitlab> gitlab # [  197.269843] setup-gitlab-start[2679]: + mkdir -p /var/lib/secrets1693vm-test-run-nixbot-gitlab> gitlab # [  197.271070] setup-gitlab-start[2679]: + echo glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle1694vm-test-run-nixbot-gitlab> gitlab # [  197.271924] setup-gitlab-start[2679]: + chmod 644 /var/lib/secrets/gitlab-token1695vm-test-run-nixbot-gitlab> gitlab # [  197.281573] setup-gitlab-start[2679]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1696vm-test-run-nixbot-gitlab> gitlab # [  197.478723] sidekiq-cluster[2137]: {"severity":"WARN","time":"2026-07-04T13:32:52.488Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9d241ea9ce50044db18d0e43","created_at":"2026-07-04T13:32:52.184Z","trace_propagation_headers":{"sentry-trace":"7b0eac1ceee846218476d54c3b673135-bc400282c6884625","baggage":"sentry-trace_id=7b0eac1ceee846218476d54c3b673135,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9YNNASMP0A4WXQMGDYMK","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9ed97a349c1bceda2326a8dad6d333d1d1ac76a447be34d4d32468d96a073f77","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:52.206Z","job_size_bytes":451,"pid":2137,"sidekiq_tid":"lynd","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9d241ea9ce50044db18d0e43: fail: 0.277776 sec","job_status":"fail","queue_duration_s":0.004036,"scheduling_latency_s":0.004102,"redis_calls":11,"redis_duration_s":0.005782,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001872,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000773283,"duration_s":0.277776,"completed_at":"2026-07-04T13:32:52.488Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.044509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1697vm-test-run-nixbot-gitlab> gitlab # [  199.210473] gitaly[1387]: time="2026-07-04T13:32:54.222Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.221" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698vm-test-run-nixbot-gitlab> gitlab # [  199.211459] gitaly[1387]: time="2026-07-04T13:32:54.223Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.221" grpc.time_ms=1.16 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699vm-test-run-nixbot-gitlab> gitlab # [  199.353569] gitaly[1387]: time="2026-07-04T13:32:54.365Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.365" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1700vm-test-run-nixbot-gitlab> gitlab # [  199.354260] gitaly[1387]: time="2026-07-04T13:32:54.365Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.365" grpc.time_ms=0.525 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1701vm-test-run-nixbot-gitlab> gitlab # [  199.398461] gitaly[1387]: time="2026-07-04T13:32:54.410Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV diskcache=9e14762c-2a92-4887-87b6-5f5ca88ae859 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T13:33:04.157" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.379" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1702vm-test-run-nixbot-gitlab> gitlab # [  199.418535] gitaly[1387]: time="2026-07-04T13:32:54.410Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T13:33:04.157" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.379" grpc.time_ms=31.613 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1703vm-test-run-nixbot-gitlab> gitlab # [  199.419709] gitaly[1387]: time="2026-07-04T13:32:54.429Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.208" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.429" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1704vm-test-run-nixbot-gitlab> gitlab # [  199.421381] gitaly[1387]: time="2026-07-04T13:32:54.430Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.208" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.429" grpc.time_ms=0.717 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1705vm-test-run-nixbot-gitlab> gitlab # [  199.483466] gitaly[1387]: time="2026-07-04T13:32:54.495Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:04.267" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.488" grpc.time_ms=7.15 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1706vm-test-run-nixbot-gitlab> gitlab # [  199.531518] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:54.542Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a5e7c1ba81b4547f04583499","created_at":"2026-07-04T13:32:54.510Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:54.537Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a5e7c1ba81b4547f04583499: start","job_status":"start","queue_duration_s":0.005635,"scheduling_latency_s":0.005672}1707vm-test-run-nixbot-gitlab> gitlab # [  199.565954] gitaly[1387]: time="2026-07-04T13:32:54.577Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.577" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1708vm-test-run-nixbot-gitlab> gitlab # [  199.568166] gitaly[1387]: time="2026-07-04T13:32:54.578Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.577" grpc.time_ms=0.293 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1709vm-test-run-nixbot-gitlab> gitlab # [  199.601524] gitaly[1387]: time="2026-07-04T13:32:54.613Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV diskcache=f0a65a1a-858b-4517-a91e-d44d6a185971 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T13:33:04.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.581" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1710vm-test-run-nixbot-gitlab> gitlab # [  199.602828] gitaly[1387]: time="2026-07-04T13:32:54.613Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T13:33:04.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.581" grpc.time_ms=32.459 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711vm-test-run-nixbot-gitlab> gitlab # [  199.651974] gitaly[1387]: time="2026-07-04T13:32:54.664Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.663" pid=1387 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1712vm-test-run-nixbot-gitlab> gitlab # [  199.653213] gitaly[1387]: time="2026-07-04T13:32:54.664Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.663" grpc.time_ms=0.471 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713vm-test-run-nixbot-gitlab> gitlab # [  199.699621] gitaly[1387]: time="2026-07-04T13:32:54.711Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:04.133" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.704" grpc.time_ms=7.286 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714vm-test-run-nixbot-gitlab> gitlab # [  199.719099] gitaly[1387]: time="2026-07-04T13:32:54.731Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:04.160" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.730" grpc.time_ms=0.153 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715vm-test-run-nixbot-gitlab> gitlab # [  199.724740] gitaly[1387]: time="2026-07-04T13:32:54.736Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T13:33:04.165" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:54.736" grpc.time_ms=0.21 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1716vm-test-run-nixbot-gitlab> gitlab # [  199.761858] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:54.773Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a5e7c1ba81b4547f04583499","created_at":"2026-07-04T13:32:54.510Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:54.537Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a5e7c1ba81b4547f04583499: done: 0.230807 sec","job_status":"done","queue_duration_s":0.005635,"scheduling_latency_s":0.005672,"redis_calls":27,"redis_duration_s":0.034939,"redis_read_bytes":152,"redis_write_bytes":3480,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.02456,"redis_feature_flag_read_bytes":144,"redis_feature_flag_write_bytes":2315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006404,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003975,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.050802825,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.230807,"completed_at":"2026-07-04T13:32:54.773Z","load_balancing_strategy":"primary","db_duration_s":0.040421,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1717vm-test-run-nixbot-gitlab> gitlab # [  199.770312] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:54.782Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5ed9d7cac37c883ad11011c1","created_at":"2026-07-04T13:32:54.543Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:54.572Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5ed9d7cac37c883ad11011c1: start","job_status":"start","queue_duration_s":0.209175,"scheduling_latency_s":0.209212}1718vm-test-run-nixbot-gitlab> gitlab # [  199.884119] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:54.895Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5ed9d7cac37c883ad11011c1","created_at":"2026-07-04T13:32:54.543Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:54.572Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5ed9d7cac37c883ad11011c1: done: 0.113949 sec","job_status":"done","queue_duration_s":0.209175,"scheduling_latency_s":0.209212,"redis_calls":14,"redis_duration_s":0.015118,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003212,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005974,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005932,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072675996,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.113949,"completed_at":"2026-07-04T13:32:54.895Z","load_balancing_strategy":"primary","db_duration_s":0.034269,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1719vm-test-run-nixbot-gitlab> gitlab # [  200.042187] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.053Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1ec902b6a5daa6a8dd6faaa9","created_at":"2026-07-04T13:32:55.044Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.051Z","job_size_bytes":5,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1ec902b6a5daa6a8dd6faaa9: start","job_status":"start","queue_duration_s":0.002442,"scheduling_latency_s":0.002478}1720vm-test-run-nixbot-gitlab> gitlab # [  200.088760] gitaly[1387]: time="2026-07-04T13:32:55.100Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPN9Z6N20P2X0E4SBWKTRYV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:05.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:55.094" grpc.time_ms=6.239 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721vm-test-run-nixbot-gitlab> gitlab # [  200.205139] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.216Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1ec902b6a5daa6a8dd6faaa9","created_at":"2026-07-04T13:32:55.044Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.051Z","job_size_bytes":5,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1ec902b6a5daa6a8dd6faaa9: done: 0.163007 sec","job_status":"done","queue_duration_s":0.002442,"scheduling_latency_s":0.002478,"redis_calls":12,"redis_duration_s":0.015029,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003761,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.009013,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000516267,"duration_s":0.163007,"completed_at":"2026-07-04T13:32:55.216Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026362,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1722vm-test-run-nixbot-gitlab> gitlab # [  200.248825] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.221Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"1ee0fb04e2794501d0717039","created_at":"2026-07-04T13:32:55.072Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.076Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-1ee0fb04e2794501d0717039: start","job_status":"start","queue_duration_s":0.144306,"scheduling_latency_s":0.144347}1723vm-test-run-nixbot-gitlab> gitlab # [  200.432512] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:55 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 31331724vm-test-run-nixbot-gitlab> gitlab # [  200.435780] setup-gitlab-start[2679]: + mkdir -p /tmp/test-flake1725vm-test-run-nixbot-gitlab> gitlab # [  200.439149] setup-gitlab-start[2679]: + cd /tmp/test-flake1726vm-test-run-nixbot-gitlab> gitlab # [  200.440817] setup-gitlab-start[2679]: + git init -b master1727vm-test-run-nixbot-gitlab> gitlab # [  200.459229] setup-gitlab-start[3002]: Initialized empty Git repository in /tmp/test-flake/.git/1728vm-test-run-nixbot-gitlab> gitlab # [  200.461386] setup-gitlab-start[2679]: + git config user.name test1729vm-test-run-nixbot-gitlab> gitlab # [  200.463106] setup-gitlab-start[2679]: + git config user.email test@example.com1730vm-test-run-nixbot-gitlab> gitlab # [  200.466605] setup-gitlab-start[2679]: + cat1731vm-test-run-nixbot-gitlab> gitlab # [  200.469705] setup-gitlab-start[2679]: + git add flake.nix1732vm-test-run-nixbot-gitlab> gitlab # [  200.477244] setup-gitlab-start[2679]: + git commit -m 'initial commit'1733vm-test-run-nixbot-gitlab> gitlab # [  200.486575] setup-gitlab-start[3007]: [master (root-commit) 2167be4] initial commit1734vm-test-run-nixbot-gitlab> gitlab # [  200.487582] setup-gitlab-start[3007]:  1 file changed, 10 insertions(+)1735vm-test-run-nixbot-gitlab> gitlab # [  200.488979] setup-gitlab-start[3007]:  create mode 100644 flake.nix1736vm-test-run-nixbot-gitlab> gitlab # [  200.491193] setup-gitlab-start[2679]: + git remote add origin http://oauth2:glpat-yOiCXOEk7SGlapyokkbB4286MQp1OjEH.01.0w1ccjsle@gitlab/root/test-flake.git1737vm-test-run-nixbot-gitlab> gitlab # [  200.494428] setup-gitlab-start[2679]: + git push -u origin master1738vm-test-run-nixbot-gitlab> gitlab # [  200.558133] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 421739vm-test-run-nixbot-gitlab> gitlab # [  200.724613] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.735Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"1ee0fb04e2794501d0717039","created_at":"2026-07-04T13:32:55.072Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.076Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-1ee0fb04e2794501d0717039: done: 0.513931 sec","job_status":"done","queue_duration_s":0.144306,"scheduling_latency_s":0.144347,"redis_calls":11,"redis_duration_s":0.01015,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003466,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.402,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.402,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.084,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000693943,"duration_s":0.513931,"completed_at":"2026-07-04T13:32:55.734Z","load_balancing_strategy":"primary","db_duration_s":0.436775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1740vm-test-run-nixbot-gitlab> gitlab # [  200.796135] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.806Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"4a1cbed39ca45023ad17723f","created_at":"2026-07-04T13:32:55.079Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.082Z","job_size_bytes":89,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-4a1cbed39ca45023ad17723f: start","job_status":"start","queue_duration_s":0.723619,"scheduling_latency_s":0.723686}1741vm-test-run-nixbot-gitlab> gitlab # [  200.877544] gitaly[1387]: time="2026-07-04T13:32:55.889Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2CKKXFTRTE1ME18F4T6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:32:55.876" grpc.time_ms=12.516 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1742vm-test-run-nixbot-gitlab> gitlab # [  200.891231] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:55.902Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"4a1cbed39ca45023ad17723f","created_at":"2026-07-04T13:32:55.079Z","trace_propagation_headers":{"sentry-trace":"b26d1645ac94484996ed8a0e951fb589-dac99e67b0c04daa","baggage":"sentry-trace_id=b26d1645ac94484996ed8a0e951fb589,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9Z6N20P2X0E4SBWKTRYV","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:55.082Z","job_size_bytes":89,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-4a1cbed39ca45023ad17723f: done: 0.096723 sec","job_status":"done","queue_duration_s":0.723619,"scheduling_latency_s":0.723686,"redis_calls":11,"redis_duration_s":0.005291,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001951,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000407035,"duration_s":0.096723,"completed_at":"2026-07-04T13:32:55.902Z","load_balancing_strategy":"primary","db_duration_s":0.030192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1743vm-test-run-nixbot-gitlab> gitlab # [  200.971708] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 4111744vm-test-run-nixbot-gitlab> gitlab # [  201.186337] gitaly[1387]: time="2026-07-04T13:32:56.198Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:06.093" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.184" grpc.time_ms=13.518 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745vm-test-run-nixbot-gitlab> gitlab # [  201.224730] gitaly[1387]: time="2026-07-04T13:32:56.236Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T13:33:26.024" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T13:32:56.224" grpc.time_ms=11.945 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746vm-test-run-nixbot-gitlab> gitlab # [  201.248811] gitaly[1387]: time="2026-07-04T13:32:56.260Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPNA2SV6TTM8T27CP66455V duration_ms=130ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1747vm-test-run-nixbot-gitlab> gitlab # [  201.275483] gitaly[1387]: time="2026-07-04T13:32:56.287Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPNA2SV6TTM8T27CP66455V duration_ms=26ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1748vm-test-run-nixbot-gitlab> gitlab # [  201.280656] gitaly[1387]: time="2026-07-04T13:32:56.287Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.128" grpc.time_ms=159.163 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1749vm-test-run-nixbot-gitlab> gitlab # [  201.330675] gitaly[1387]: time="2026-07-04T13:32:56.342Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.341" grpc.time_ms=0.6 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1750vm-test-run-nixbot-gitlab> gitlab # [  201.371507] gitaly[1387]: time="2026-07-04T13:32:56.383Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.383" grpc.time_ms=0.523 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1751vm-test-run-nixbot-gitlab> gitlab # [  201.411114] gitaly[1387]: time="2026-07-04T13:32:56.423Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.422" grpc.time_ms=0.295 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1752vm-test-run-nixbot-gitlab> gitlab # [  201.488512] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:56.498Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2167be41b58e6269099e2a2588e308eb35c22060 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2e1b7339b3b47883b9c3e004","created_at":"2026-07-04T13:32:56.491Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-b42bf89324a84b53","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.496Z","job_size_bytes":185,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2e1b7339b3b47883b9c3e004: start","job_status":"start","queue_duration_s":0.001469,"scheduling_latency_s":0.001507}1753vm-test-run-nixbot-gitlab> gitlab # [  201.528456] gitaly[1387]: time="2026-07-04T13:32:56.540Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:06.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.534" grpc.time_ms=6.29 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754vm-test-run-nixbot-gitlab> gitlab # [  201.530684] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:56.542Z","message":"changes: \"0000000000000000000000000000000000000000 2167be41b58e6269099e2a2588e308eb35c22060 refs/heads/master\\n\""}1755vm-test-run-nixbot-gitlab> gitlab # [  201.581432] gitaly[1387]: time="2026-07-04T13:32:56.593Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1558 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T13:33:06.046" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.576" grpc.time_ms=16.762 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756vm-test-run-nixbot-gitlab> gitlab # [  201.599486] gitaly[1387]: time="2026-07-04T13:32:56.611Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:06.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.603" grpc.time_ms=7.791 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757vm-test-run-nixbot-gitlab> gitlab # [  201.613221] gitaly[1387]: time="2026-07-04T13:32:56.625Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:06.017" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.617" grpc.time_ms=6.792 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758vm-test-run-nixbot-gitlab> gitlab # [  201.632871] gitaly[1387]: time="2026-07-04T13:32:56.645Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1570 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T13:33:06.027" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.627" grpc.time_ms=17.256 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759vm-test-run-nixbot-gitlab> gitlab # [  201.668883] gitaly[1387]: time="2026-07-04T13:32:56.680Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T13:33:06.068" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.668" grpc.time_ms=11.409 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760vm-test-run-nixbot-gitlab> gitlab # [  201.689704] gitaly[1387]: time="2026-07-04T13:32:56.701Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPNA2SV6TTM8T27CP66455V duration_ms=237ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1761vm-test-run-nixbot-gitlab> gitlab # [  201.691092] gitaly[1387]: time="2026-07-04T13:32:56.703Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.463" grpc.time_ms=239.667 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1762vm-test-run-nixbot-gitlab> gitlab # [  201.720101] gitaly[1387]: time="2026-07-04T13:32:56.732Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.731" grpc.time_ms=0.352 pid=1387 span.kind=server system=grpc1763vm-test-run-nixbot-gitlab> gitlab # [  201.738429] gitaly[1387]: time="2026-07-04T13:32:56.750Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V diskcache=9f0fa52a-2129-4227-bf24-f303c20b69f3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:32:56.059" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1764vm-test-run-nixbot-gitlab> gitlab # [  201.754875] gitaly[1387]: time="2026-07-04T13:32:56.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=221 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13473 command.oublock=32 command.real_time_ms=665 command.spawn_token_fork_ms=0 command.system_time_ms=147 command.user_time_ms=74 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:32:56.059" grpc.time_ms=690.813 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1765vm-test-run-nixbot-gitlab> gitlab # [  201.757168] gitaly[1387]: time="2026-07-04T13:32:56.769Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:32:56.768" grpc.time_ms=0.366 pid=1387 span.kind=server system=grpc1766vm-test-run-nixbot-gitlab> gitlab # [  201.766499] gitaly[1387]: time="2026-07-04T13:32:56.778Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V diskcache=cce0d4ed-0b5f-42d6-a8cb-43c400b7d196 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:06.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.693" pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1767vm-test-run-nixbot-gitlab> gitlab # [  201.768963] gitaly[1387]: time="2026-07-04T13:32:56.779Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4854 command.oublock=8 command.real_time_ms=78 command.spawn_token_fork_ms=10 command.system_time_ms=40 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:06.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.693" grpc.time_ms=85.595 pid=1387 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1768vm-test-run-nixbot-gitlab> gitlab # [  201.783929] gitaly[1387]: time="2026-07-04T13:32:56.795Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T13:33:06.195" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.795" grpc.time_ms=0.141 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769vm-test-run-nixbot-gitlab> gitlab # [  201.797330] gitaly[1387]: time="2026-07-04T13:32:56.809Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=938 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T13:33:06.199" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.800" grpc.time_ms=9.293 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1770vm-test-run-nixbot-gitlab> gitlab # [  201.853260] gitaly[1387]: time="2026-07-04T13:32:56.865Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-04T13:33:26.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:56.860" grpc.time_ms=5.204 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1771vm-test-run-nixbot-gitlab> gitlab # [  201.872823] gitaly[1387]: time="2026-07-04T13:32:56.884Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T13:33:06.284" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:56.884" grpc.time_ms=0.21 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772vm-test-run-nixbot-gitlab> gitlab # [  201.879953] gitaly[1387]: time="2026-07-04T13:32:56.890Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T13:33:06.286" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T13:32:56.886" grpc.time_ms=3.956 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773vm-test-run-nixbot-gitlab> gitlab # [  201.896537] gitaly[1387]: time="2026-07-04T13:32:56.908Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:06.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.902" grpc.time_ms=6.055 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774vm-test-run-nixbot-gitlab> gitlab # [  201.942205] gitaly[1387]: time="2026-07-04T13:32:56.921Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:06.013" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:56.913" grpc.time_ms=7.781 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775vm-test-run-nixbot-gitlab> gitlab # [  201.973831] gitlab-workhorse[1288]: time="2026-07-04T13:32:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521776vm-test-run-nixbot-gitlab> gitlab # [  201.974373] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:56 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 9891777vm-test-run-nixbot-gitlab> gitlab # [  201.980663] setup-gitlab-start[3011]: To http://gitlab/root/test-flake.git1778vm-test-run-nixbot-gitlab> gitlab # [  201.981269] setup-gitlab-start[3011]:  * [new branch]      master -> master1779vm-test-run-nixbot-gitlab> gitlab # [  201.986381] setup-gitlab-start[3011]: branch 'master' set up to track 'origin/master'.1780vm-test-run-nixbot-gitlab> gitlab # [  201.990666] systemd[1]: Finished setup-gitlab.service.1781vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 202.96 seconds)1782vm-test-run-nixbot-gitlab> gitlab # [  201.993675] gitaly[1387]: time="2026-07-04T13:32:57.005Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T13:33:26.194" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:56.995" grpc.time_ms=10.573 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1784vm-test-run-nixbot-gitlab> gitlab # [  202.018040] systemd[1]: Starting nixbot CI service...1785vm-test-run-nixbot-gitlab> gitlab # [  202.477546] gitaly[1387]: time="2026-07-04T13:32:57.489Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T13:33:07.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.487" grpc.time_ms=1.487 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786vm-test-run-nixbot-gitlab> gitlab # [  202.552712] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.563Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2167be41b58e6269099e2a2588e308eb35c22060 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2e1b7339b3b47883b9c3e004","created_at":"2026-07-04T13:32:56.491Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-b42bf89324a84b53","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.496Z","job_size_bytes":185,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2e1b7339b3b47883b9c3e004: done: 1.06563 sec","job_status":"done","queue_duration_s":0.001469,"scheduling_latency_s":0.001507,"gitaly_calls":14,"gitaly_duration_s":0.304447,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.057645,"redis_read_bytes":1442,"redis_write_bytes":19008,"redis_cache_calls":14,"redis_cache_duration_s":0.004308,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.01006,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001775,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4933,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003794,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.019751,"redis_repository_cache_read_bytes":230,"redis_repository_cache_write_bytes":6310,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.017957,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.363,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.378,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.086,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.443957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001042591,"exclusive_lock_hold_duration_s":0.039143522,"duration_s":1.06563,"completed_at":"2026-07-04T13:32:57.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.117067,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1787vm-test-run-nixbot-gitlab> gitlab # [  202.563299] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.566Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"547824222ea9dda990a64897","created_at":"2026-07-04T13:32:56.647Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.652Z","job_size_bytes":46,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-547824222ea9dda990a64897: start","job_status":"start","queue_duration_s":0.914732,"scheduling_latency_s":0.914773}1788vm-test-run-nixbot-gitlab> gitlab # [  202.599235] gitaly[1387]: time="2026-07-04T13:32:57.610Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-04T19:32:57.607" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:32:57.608" grpc.time_ms=2.692 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789vm-test-run-nixbot-gitlab> gitlab # [  202.691944] gitaly[1387]: time="2026-07-04T13:32:57.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=82 command.spawn_token_fork_ms=76 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T13:33:27.018" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.618" grpc.time_ms=84.619 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790vm-test-run-nixbot-gitlab> gitlab # [  202.733391] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.744Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"547824222ea9dda990a64897","created_at":"2026-07-04T13:32:56.647Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.652Z","job_size_bytes":46,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-547824222ea9dda990a64897: done: 0.177437 sec","job_status":"done","queue_duration_s":0.914732,"scheduling_latency_s":0.914773,"gitaly_calls":2,"gitaly_duration_s":0.092821,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011883,"redis_read_bytes":68,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002177,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001257,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003211,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003783,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001455,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000959619,"duration_s":0.177437,"completed_at":"2026-07-04T13:32:57.744Z","load_balancing_strategy":"primary","db_duration_s":0.015688,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1791vm-test-run-nixbot-gitlab> gitlab # [  202.773136] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.747Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"01ee1246df0a291b74043dee","created_at":"2026-07-04T13:32:56.661Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.664Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-01ee1246df0a291b74043dee: start","job_status":"start","queue_duration_s":1.082533,"scheduling_latency_s":1.082572}1792vm-test-run-nixbot-gitlab> gitlab # [  202.802094] gitaly[1387]: time="2026-07-04T13:32:57.814Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V diskcache=6230a79d-07c7-404f-91b8-a50c7e6e0bac grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T19:32:57.789" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.789" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793vm-test-run-nixbot-gitlab> gitlab # [  202.803517] gitaly[1387]: time="2026-07-04T13:32:57.814Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1276 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T19:32:57.789" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.789" grpc.time_ms=24.357 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1794vm-test-run-nixbot-gitlab> gitlab # [  202.831320] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.843Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"01ee1246df0a291b74043dee","created_at":"2026-07-04T13:32:56.661Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.664Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-01ee1246df0a291b74043dee: done: 0.095686 sec","job_status":"done","queue_duration_s":1.082533,"scheduling_latency_s":1.082572,"gitaly_calls":1,"gitaly_duration_s":0.026756,"redis_calls":15,"redis_duration_s":0.007459,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001655,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000696,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003316,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.054676782,"duration_s":0.095686,"completed_at":"2026-07-04T13:32:57.843Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1795vm-test-run-nixbot-gitlab> gitlab # [  202.908318] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.844Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"09ae8b7cfcfcde3b077c615f","created_at":"2026-07-04T13:32:56.791Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.793Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-09ae8b7cfcfcde3b077c615f: start","job_status":"start","queue_duration_s":1.051516,"scheduling_latency_s":1.051537}1796vm-test-run-nixbot-gitlab> gitlab # [  202.909855] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.881Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"09ae8b7cfcfcde3b077c615f","created_at":"2026-07-04T13:32:56.791Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:56.793Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-09ae8b7cfcfcde3b077c615f: done: 0.036499 sec","job_status":"done","queue_duration_s":1.051516,"scheduling_latency_s":1.051537,"gitaly_calls":1,"gitaly_duration_s":0.008982,"redis_calls":16,"redis_duration_s":0.003683,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.00026,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001543,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000716,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036499,"completed_at":"2026-07-04T13:32:57.881Z","load_balancing_strategy":"primary","db_duration_s":0.002031,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1797vm-test-run-nixbot-gitlab> gitlab # [  202.986983] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:57.882Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fdf497fedaf7c1df4efd8e90","created_at":"2026-07-04T13:32:57.419Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b5c45a96d7f132c283416245e9e1b02e0e22be6cbf225e33e35bd0e84c666aea","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:57.430Z","job_size_bytes":467,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fdf497fedaf7c1df4efd8e90: start","job_status":"start","queue_duration_s":0.451552,"scheduling_latency_s":0.451575}1798vm-test-run-nixbot-gitlab> gitlab # [  203.061640] gitaly[1387]: time="2026-07-04T13:32:57.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T13:33:27.266" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.867" grpc.time_ms=6.834 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [  203.062895] gitaly[1387]: time="2026-07-04T13:32:57.915Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:27.313" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:57.914" grpc.time_ms=0.662 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [  203.063754] gitaly[1387]: time="2026-07-04T13:32:57.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T13:33:07.018" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:57.918" grpc.time_ms=5.971 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801vm-test-run-nixbot-gitlab> gitlab # [  203.067276] gitaly[1387]: time="2026-07-04T13:32:58.005Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:28.003" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:58.003" grpc.time_ms=1.263 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802vm-test-run-nixbot-gitlab> gitlab # [  203.158225] gitaly[1387]: time="2026-07-04T13:32:58.010Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=79 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T13:33:28.007" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:58.007" grpc.time_ms=2.724 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803vm-test-run-nixbot-gitlab> gitlab # [  203.159745] gitaly[1387]: time="2026-07-04T13:32:58.095Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:08.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:32:58.085" grpc.time_ms=10.095 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804vm-test-run-nixbot-gitlab> gitlab # [  203.328951] nixbot[3145]: {"ts": "2026-07-04T13:32:58.340Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1805vm-test-run-nixbot-gitlab> gitlab # [  203.358344] nixbot[3145]: {"ts": "2026-07-04T13:32:58.370Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1806vm-test-run-nixbot-gitlab> gitlab # [  203.394288] nixbot[3145]: {"ts": "2026-07-04T13:32:58.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1807vm-test-run-nixbot-gitlab> gitlab # [  203.399101] nixbot[3145]: {"ts": "2026-07-04T13:32:58.410Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1808vm-test-run-nixbot-gitlab> gitlab # [  203.402817] nixbot[3145]: {"ts": "2026-07-04T13:32:58.414Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1809vm-test-run-nixbot-gitlab> gitlab # [  203.407814] nixbot[3145]: {"ts": "2026-07-04T13:32:58.419Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1810vm-test-run-nixbot-gitlab> gitlab # [  203.413306] nixbot[3145]: {"ts": "2026-07-04T13:32:58.425Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1811vm-test-run-nixbot-gitlab> gitlab # [  203.416673] nixbot[3145]: {"ts": "2026-07-04T13:32:58.428Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1812vm-test-run-nixbot-gitlab> gitlab # [  203.424813] nixbot[3145]: {"ts": "2026-07-04T13:32:58.436Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1813vm-test-run-nixbot-gitlab> gitlab # [  203.428157] nixbot[3145]: {"ts": "2026-07-04T13:32:58.438Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1814vm-test-run-nixbot-gitlab> gitlab # [  203.431353] nixbot[3145]: {"ts": "2026-07-04T13:32:58.443Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1815vm-test-run-nixbot-gitlab> gitlab # [  203.435936] nixbot[3145]: {"ts": "2026-07-04T13:32:58.447Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1816vm-test-run-nixbot-gitlab> gitlab # [  203.438787] nixbot[3145]: {"ts": "2026-07-04T13:32:58.450Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1817vm-test-run-nixbot-gitlab> gitlab # [  203.441685] nixbot[3145]: {"ts": "2026-07-04T13:32:58.452Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1818vm-test-run-nixbot-gitlab> gitlab # [  203.444661] nixbot[3145]: {"ts": "2026-07-04T13:32:58.455Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1819vm-test-run-nixbot-gitlab> gitlab # [  203.447498] nixbot[3145]: {"ts": "2026-07-04T13:32:58.457Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1820vm-test-run-nixbot-gitlab> gitlab # [  203.449760] nixbot[3145]: {"ts": "2026-07-04T13:32:58.459Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1821vm-test-run-nixbot-gitlab> gitlab # [  203.452343] nixbot[3145]: {"ts": "2026-07-04T13:32:58.461Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1822vm-test-run-nixbot-gitlab> gitlab # [  203.453274] nixbot[3145]: {"ts": "2026-07-04T13:32:58.465Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1823vm-test-run-nixbot-gitlab> gitlab # [  203.455351] nixbot[3145]: {"ts": "2026-07-04T13:32:58.467Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1824vm-test-run-nixbot-gitlab> gitlab # [  203.457451] nixbot[3145]: {"ts": "2026-07-04T13:32:58.469Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1825vm-test-run-nixbot-gitlab> gitlab # [  203.743237] gitaly[1387]: time="2026-07-04T13:32:58.754Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T13:33:28.135" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:58.735" grpc.time_ms=18.758 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1826vm-test-run-nixbot-gitlab> gitlab # [  203.764176] gitaly[1387]: time="2026-07-04T13:32:58.775Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA2SV6TTM8T27CP66455V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T13:33:28.159" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:58.760" grpc.time_ms=14.697 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827vm-test-run-nixbot-gitlab> gitlab # [  203.776798] nixbot[3145]: INFO:     Started server process [3145]1828vm-test-run-nixbot-gitlab> gitlab # [  203.781208] nixbot[3145]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1829vm-test-run-nixbot-gitlab> gitlab # [  203.782299] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60272 - "GET /health HTTP/1.1" 200 OK1830vm-test-run-nixbot-gitlab> gitlab # [  203.819332] systemd[1]: Started nixbot CI service.1831vm-test-run-nixbot-gitlab> gitlab # [  203.820483] systemd[1]: Reached target Multi-User System.1832vm-test-run-nixbot-gitlab> gitlab # [  203.821161] systemd[1]: Startup finished in 1.601s (kernel) + 4.287s (initrd) + 3min 17.930s (userspace) = 3min 23.819s.1833vm-test-run-nixbot-gitlab> gitlab # [  203.825153] sidekiq-cluster[2137]: {"severity":"WARN","time":"2026-07-04T13:32:58.834Z","class":"CreatePipelineWorker","correlation_id":"01KWPNA2SV6TTM8T27CP66455V","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"2167be41b58e6269099e2a2588e308eb35c22060","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"2167be41b58e6269099e2a2588e308eb35c22060","ref":"refs/heads/master"},"retry":0}1834vm-test-run-nixbot-gitlab> gitlab # [  203.828378] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:58.840Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fdf497fedaf7c1df4efd8e90","created_at":"2026-07-04T13:32:57.419Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-0a7dc2357afd4339","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b5c45a96d7f132c283416245e9e1b02e0e22be6cbf225e33e35bd0e84c666aea","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:57.430Z","job_size_bytes":467,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fdf497fedaf7c1df4efd8e90: done: 0.957572 sec","job_status":"done","queue_duration_s":0.451552,"scheduling_latency_s":0.451575,"gitaly_calls":7,"gitaly_duration_s":0.074256,"redis_calls":53,"redis_duration_s":0.016673,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000995,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.0101,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003227,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.144,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.139,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.675897,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000332444,"duration_s":0.957572,"completed_at":"2026-07-04T13:32:58.840Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.17598,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1835vm-test-run-nixbot-gitlab> gitlab # [  203.917263] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:58.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6c7357439289d0f858d35f80","created_at":"2026-07-04T13:32:57.717Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-ebf67ac11cd34877","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:57.722Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6c7357439289d0f858d35f80: start","job_status":"start","queue_duration_s":1.206186,"scheduling_latency_s":1.206224}1836vm-test-run-nixbot-gitlab> gitlab # [  203.965924] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:58.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6c7357439289d0f858d35f80","created_at":"2026-07-04T13:32:57.717Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-ebf67ac11cd34877","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:57.722Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6c7357439289d0f858d35f80: done: 0.048253 sec","job_status":"done","queue_duration_s":1.206186,"scheduling_latency_s":1.206224,"redis_calls":19,"redis_duration_s":0.005399,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001028,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000552,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002808,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006805056,"duration_s":0.048253,"completed_at":"2026-07-04T13:32:58.977Z","load_balancing_strategy":"primary","db_duration_s":0.006026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1837vm-test-run-nixbot-gitlab> gitlab # [  204.003791] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:58.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"392c31befa73528fce3b83b4","created_at":"2026-07-04T13:32:58.957Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-a342653973364f79","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:58.961Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-392c31befa73528fce3b83b4: start","job_status":"start","queue_duration_s":0.01898,"scheduling_latency_s":0.019029}1838vm-test-run-nixbot-gitlab> gitlab # [  204.109201] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:32:59.120Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"392c31befa73528fce3b83b4","created_at":"2026-07-04T13:32:58.957Z","trace_propagation_headers":{"sentry-trace":"ec21e7f2c32843e5997b9303b35942ad-a342653973364f79","baggage":"sentry-trace_id=ec21e7f2c32843e5997b9303b35942ad,sentry-release=Unknown"},"correlation_id":"01KWPNA2SV6TTM8T27CP66455V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T13:32:58.961Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-392c31befa73528fce3b83b4: done: 0.140187 sec","job_status":"done","queue_duration_s":0.01898,"scheduling_latency_s":0.019029,"redis_calls":11,"redis_duration_s":0.0037679999999999996,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002459,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.061,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000497829,"duration_s":0.140187,"completed_at":"2026-07-04T13:32:59.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.068847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1839vm-test-run-nixbot-gitlab> gitlab # [  204.119967] gitaly[1387]: time="2026-07-04T13:32:59.131Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA5JG6WXD4JP8XFCVJG78 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T13:33:29.019" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:32:59.120" grpc.time_ms=10.454 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840vm-test-run-nixbot-gitlab> gitlab # [  204.192296] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:59 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3231841vm-test-run-nixbot-gitlab> gitlab # [  204.193916] nixbot[3145]: {"ts": "2026-07-04T13:32:59.158Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1842vm-test-run-nixbot-gitlab> gitlab # [  204.219145] nixbot[3145]: {"ts": "2026-07-04T13:32:59.231Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1843vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.13 seconds)1844vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1845vm-test-run-nixbot-gitlab> gitlab # [  204.269377] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60822 - "GET /health HTTP/1.1" 200 OK1846vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds)1847vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1848vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1849vm-test-run-nixbot-gitlab> gitlab # [  204.297846] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 621850vm-test-run-nixbot-gitlab> gitlab # [  204.304086] nixbot[3145]: {"ts": "2026-07-04T13:32:59.316Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1851vm-test-run-nixbot-gitlab> gitlab # [  204.309507] nixbot[3145]: {"ts": "2026-07-04T13:32:59.321Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1852vm-test-run-nixbot-gitlab> gitlab # [  204.348077] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 561853vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds)1854vm-test-run-nixbot-gitlab> []1855vm-test-run-nixbot-gitlab> gitlab # [  204.457248] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:32:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1451856vm-test-run-nixbot-gitlab> gitlab # [  204.467623] nixbot[3145]: {"ts": "2026-07-04T13:32:59.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1857vm-test-run-nixbot-gitlab> gitlab # [  204.996464] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 5271858vm-test-run-nixbot-gitlab> gitlab # [  205.006873] nixbot[3145]: {"ts": "2026-07-04T13:33:00.009Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1859vm-test-run-nixbot-gitlab> gitlab # [  205.091651] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 781860vm-test-run-nixbot-gitlab> gitlab # [  205.101674] nixbot[3145]: {"ts": "2026-07-04T13:33:00.113Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1861vm-test-run-nixbot-gitlab> gitlab # [  205.214503] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 951862vm-test-run-nixbot-gitlab> gitlab # [  205.224123] nixbot[3145]: {"ts": "2026-07-04T13:33:00.235Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1863vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1864vm-test-run-nixbot-gitlab> gitlab # [  205.336360] gitaly[1387]: time="2026-07-04T13:33:00.347Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA6Z0MWBH20EEVGZFW8NH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-04T13:33:30.130" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:00.331" grpc.time_ms=15.436 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865vm-test-run-nixbot-gitlab> gitlab # [  205.393594] gitaly[1387]: time="2026-07-04T13:33:00.359Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA6Z0MWBH20EEVGZFW8NH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:30.157" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:00.358" grpc.time_ms=0.625 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866vm-test-run-nixbot-gitlab> gitlab # [  205.414883] gitaly[1387]: time="2026-07-04T13:33:00.378Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA6Z0MWBH20EEVGZFW8NH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T13:33:10.141" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:00.362" grpc.time_ms=15.034 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867vm-test-run-nixbot-gitlab> gitlab # [  205.436503] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 1841868vm-test-run-nixbot-gitlab> gitlab # [  205.440474] nixbot[3145]: {"ts": "2026-07-04T13:33:00.442Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1869vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1870vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-04T13:32:59.455Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1871vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.19 seconds)1872vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1873vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1874vm-test-run-nixbot-gitlab> gitlab # [  205.459079] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 431875vm-test-run-nixbot-gitlab> gitlab # [  205.500547] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 661876vm-test-run-nixbot-gitlab> gitlab # [  205.506610] nixbot[3145]: {"ts": "2026-07-04T13:33:00.518Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1877vm-test-run-nixbot-gitlab> gitlab # [  205.511137] nixbot[3145]: {"ts": "2026-07-04T13:33:00.521Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1878vm-test-run-nixbot-gitlab> gitlab # [  205.516617] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 241879vm-test-run-nixbot-gitlab> gitlab # [  205.592218] gitaly[1387]: time="2026-07-04T13:33:00.603Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA78BMQ8GYAQ22TPKJQ3R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.589" grpc.time_ms=13.762 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1880vm-test-run-nixbot-gitlab> gitlab # [  205.613547] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 491881vm-test-run-nixbot-gitlab> gitlab # [  205.620464] gitaly[1387]: time="2026-07-04T13:33:00.632Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=19 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA77JAP384QAEBS8NES35 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.606" grpc.time_ms=25.836 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1882vm-test-run-nixbot-gitlab> gitlab # [  205.642732] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.54.0" 1121883vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1884vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1885vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1886vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1887vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1888vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1889vm-test-run-nixbot-gitlab> gitlab # [  205.701672] gitaly[1387]: time="2026-07-04T13:33:00.713Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7A2MTHTHW7ZFR14JBFY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.697" pid=1387 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1890vm-test-run-nixbot-gitlab> gitlab # [  205.731738] gitaly[1387]: time="2026-07-04T13:33:00.713Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7A2MTHTHW7ZFR14JBFY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.697" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1891vm-test-run-nixbot-gitlab> gitlab # [  205.753317] gitlab-workhorse[1288]: time="2026-07-04T13:33:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491892vm-test-run-nixbot-gitlab> gitlab # [  205.756842] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 1041893vm-test-run-nixbot-gitlab> gitlab # [  205.760401] gitaly[1387]: time="2026-07-04T13:33:00.714Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7A2MTHTHW7ZFR14JBFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.697" grpc.time_ms=16.715 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1894vm-test-run-nixbot-gitlab> gitlab # [  205.798549] gitaly[1387]: time="2026-07-04T13:33:00.810Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7DFENVV54YESQG45D98 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.791" pid=1387 remote_ip="2001:db8:1::1" request_sha=b70976970a1cb50c4bbd8ab537ffc3ceafcd4836 response_bytes=378 span.kind=server system=grpc1895vm-test-run-nixbot-gitlab> gitlab # [  205.815164] gitaly[1387]: time="2026-07-04T13:33:00.810Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1232 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7DFENVV54YESQG45D98 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.791" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1896vm-test-run-nixbot-gitlab> gitlab # [  205.837250] gitlab-workhorse[1288]: time="2026-07-04T13:33:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3781897vm-test-run-nixbot-gitlab> gitlab # [  205.840700] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 378 "" "git/2.54.0" 921898vm-test-run-nixbot-gitlab> gitlab # [  205.844313] gitaly[1387]: time="2026-07-04T13:33:00.811Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1232 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7DFENVV54YESQG45D98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.791" grpc.time_ms=19.977 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1899vm-test-run-nixbot-gitlab> gitlab # [  205.899414] gitaly[1387]: time="2026-07-04T13:33:00.911Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T13:33:10.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:00.903" grpc.time_ms=7.183 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900vm-test-run-nixbot-gitlab> gitlab # [  205.932510] gitaly[1387]: time="2026-07-04T13:33:00.944Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T13:33:30.729" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T13:33:00.930" grpc.time_ms=14.163 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901vm-test-run-nixbot-gitlab> gitlab # [  205.945168] gitaly[1387]: time="2026-07-04T13:33:00.957Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPNA7C17H75WN3R7S7CRQRE duration_ms=105ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/allowed"1902vm-test-run-nixbot-gitlab> gitlab # [  205.947631] gitaly[1387]: time="2026-07-04T13:33:00.959Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7JB2YV1ZV7TQS5GD05G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.944" grpc.time_ms=14.494 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1903vm-test-run-nixbot-gitlab> gitlab # [  205.951217] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 851904vm-test-run-nixbot-gitlab> gitlab # [  205.971245] gitaly[1387]: time="2026-07-04T13:33:00.982Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPNA7C17H75WN3R7S7CRQRE duration_ms=24ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/pre_receive"1905vm-test-run-nixbot-gitlab> gitlab # [  205.972164] gitaly[1387]: time="2026-07-04T13:33:00.983Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:00.851" grpc.time_ms=131.558 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1906vm-test-run-nixbot-gitlab> gitlab # [  206.008156] gitaly[1387]: time="2026-07-04T13:33:01.019Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.019" grpc.time_ms=0.575 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1907vm-test-run-nixbot-gitlab> gitlab # [  206.015103] gitaly[1387]: time="2026-07-04T13:33:01.026Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7N33XEEJAYX7CNXDKRV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:01.010" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1908vm-test-run-nixbot-gitlab> gitlab # [  206.016855] gitaly[1387]: time="2026-07-04T13:33:01.027Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7N33XEEJAYX7CNXDKRV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:01.010" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1909vm-test-run-nixbot-gitlab> gitlab # [  206.018193] gitlab-workhorse[1288]: time="2026-07-04T13:33:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671910vm-test-run-nixbot-gitlab> gitlab # [  206.019112] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 641911vm-test-run-nixbot-gitlab> gitlab # [  206.019859] gitaly[1387]: time="2026-07-04T13:33:01.027Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7N33XEEJAYX7CNXDKRV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:01.010" grpc.time_ms=16.921 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1912vm-test-run-nixbot-gitlab> gitlab # [  206.040830] gitaly[1387]: time="2026-07-04T13:33:01.052Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.052" grpc.time_ms=0.33 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1913vm-test-run-nixbot-gitlab> gitlab # [  206.117941] gitaly[1387]: time="2026-07-04T13:33:01.129Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.129" grpc.time_ms=0.427 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1914vm-test-run-nixbot-gitlab> gitlab # [  206.161439] gitaly[1387]: time="2026-07-04T13:33:01.173Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:31.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.172" grpc.time_ms=0.738 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915vm-test-run-nixbot-gitlab> gitlab # [  206.193616] gitaly[1387]: time="2026-07-04T13:33:01.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-04T13:33:31.097" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.198" grpc.time_ms=7.028 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916vm-test-run-nixbot-gitlab> gitlab # [  206.218229] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.229Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2167be41b58e6269099e2a2588e308eb35c22060 fe2a76c9ff364b509bf59bd688e036c631d88243 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e762f541a090b2c3aaff1b9f","created_at":"2026-07-04T13:33:01.204Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-af757cf0875f4ee4","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.207Z","job_size_bytes":185,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e762f541a090b2c3aaff1b9f: start","job_status":"start","queue_duration_s":0.022543,"scheduling_latency_s":0.022584}1917vm-test-run-nixbot-gitlab> gitlab # [  206.241369] gitaly[1387]: time="2026-07-04T13:33:01.253Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:11.156" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.246" grpc.time_ms=6.741 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918vm-test-run-nixbot-gitlab> gitlab # [  206.245110] gitaly[1387]: time="2026-07-04T13:33:01.256Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPNA7C17H75WN3R7S7CRQRE duration_ms=84ns method=POST pid=1387 status=200 url="http://unix/api/v4/internal/post_receive"1919vm-test-run-nixbot-gitlab> gitlab # [  206.246359] gitaly[1387]: time="2026-07-04T13:33:01.257Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.170" grpc.time_ms=86.959 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1920vm-test-run-nixbot-gitlab> gitlab # [  206.298883] gitaly[1387]: time="2026-07-04T13:33:01.310Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE diskcache=a5528e82-ec26-4082-bef6-ae90116bab9f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.749" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1921vm-test-run-nixbot-gitlab> gitlab # [  206.299802] gitlab-workhorse[1288]: time="2026-07-04T13:33:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521922vm-test-run-nixbot-gitlab> gitlab # [  206.300833] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:01 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 6371923vm-test-run-nixbot-gitlab> gitlab # [  206.302067] gitaly[1387]: time="2026-07-04T13:33:01.310Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=187 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13201 command.oublock=32 command.real_time_ms=549 command.spawn_token_fork_ms=2 command.system_time_ms=122 command.user_time_ms=65 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:00.749" grpc.time_ms=560.474 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1924vm-test-run-nixbot-gitlab> gitlab # [  206.321205] gitaly[1387]: time="2026-07-04T13:33:01.333Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T13:33:11.242" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:01.332" grpc.time_ms=0.261 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925vm-test-run-nixbot-gitlab> gitlab # [  206.349229] gitaly[1387]: time="2026-07-04T13:33:01.360Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=617 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T13:33:11.265" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.355" grpc.time_ms=5.433 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1927vm-test-run-nixbot-gitlab> gitlab #    2167be4..fe2a76c  master -> master1928vm-test-run-nixbot-gitlab> gitlab # [  206.385053] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.396Z","message":"changes: \"2167be41b58e6269099e2a2588e308eb35c22060 fe2a76c9ff364b509bf59bd688e036c631d88243 refs/heads/master\\n\""}1929vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.94 seconds)1930vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1931vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1932vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'1933vm-test-run-nixbot-gitlab> gitlab # [  206.410165] gitaly[1387]: time="2026-07-04T13:33:01.422Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:11.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.413" grpc.time_ms=8.077 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [  206.411095] gitaly[1387]: time="2026-07-04T13:33:01.422Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.421" grpc.time_ms=0.721 pid=1387 span.kind=server system=grpc1935vm-test-run-nixbot-gitlab> gitlab # [  206.464521] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60836 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK1936vm-test-run-nixbot-gitlab> gitlab # [  206.467982] gitaly[1387]: time="2026-07-04T13:33:01.480Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T13:33:11.054" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:01.465" grpc.time_ms=8.248 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [  206.472958] gitaly[1387]: time="2026-07-04T13:33:01.485Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:01.484" grpc.time_ms=0.363 pid=1387 span.kind=server system=grpc1938vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.10 seconds)1939vm-test-run-nixbot-gitlab> []1940vm-test-run-nixbot-gitlab> gitlab # [  206.508399] gitaly[1387]: time="2026-07-04T13:33:01.520Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T13:33:11.077" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.487" grpc.time_ms=32.78 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1941vm-test-run-nixbot-gitlab> gitlab # [  206.529991] gitaly[1387]: time="2026-07-04T13:33:01.542Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=11 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=11 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-04T13:33:31.524" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.524" grpc.time_ms=17.293 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1942vm-test-run-nixbot-gitlab> gitlab # [  206.556491] gitaly[1387]: time="2026-07-04T13:33:01.568Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T diskcache=fd57f109-c7d7-4b05-a8ad-4b242cb574d8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:11.273" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:01.364" pid=1387 ref=refs/keep-around/2167be41b58e6269099e2a2588e308eb35c22060 remote_ip="2001:db8:1::1" revision=2167be41b58e6269099e2a2588e308eb35c22060 span.kind=server system=grpc username=root1943vm-test-run-nixbot-gitlab> gitlab # [  206.557058] gitaly[1387]: time="2026-07-04T13:33:01.568Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=97 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5173 command.oublock=8 command.real_time_ms=153 command.spawn_token_fork_ms=0 command.system_time_ms=77 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:11.273" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:01.364" grpc.time_ms=204.509 pid=1387 ref=refs/keep-around/2167be41b58e6269099e2a2588e308eb35c22060 remote_ip="2001:db8:1::1" revision=2167be41b58e6269099e2a2588e308eb35c22060 span.kind=server system=grpc username=root1944vm-test-run-nixbot-gitlab> gitlab # [  206.560953] gitaly[1387]: time="2026-07-04T13:33:01.573Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:31.372" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.572" grpc.time_ms=0.32 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945vm-test-run-nixbot-gitlab> gitlab # [  206.568803] gitaly[1387]: time="2026-07-04T13:33:01.580Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T13:33:11.166" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T13:33:01.576" grpc.time_ms=4.011 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946vm-test-run-nixbot-gitlab> gitlab # [  206.582728] gitaly[1387]: time="2026-07-04T13:33:01.594Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:11.178" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:01.588" grpc.time_ms=5.755 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1947vm-test-run-nixbot-gitlab> gitlab # [  206.691644] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 5771948vm-test-run-nixbot-gitlab> gitlab # [  206.696662] nixbot[3145]: {"ts": "2026-07-04T13:33:01.704Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 \"HTTP/1.1 201 Created\""}1949vm-test-run-nixbot-gitlab> gitlab # [  206.704778] nixbot[3145]: {"ts": "2026-07-04T13:33:01.716Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-2167be41b58e-94e111c4", "argv0": "bwrap"}1950vm-test-run-nixbot-gitlab> gitlab # [  206.738129] gitaly[1387]: time="2026-07-04T13:33:01.750Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T13:33:11.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.747" grpc.time_ms=2.524 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951vm-test-run-nixbot-gitlab> gitlab # [  206.782152] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.793Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2167be41b58e6269099e2a2588e308eb35c22060 fe2a76c9ff364b509bf59bd688e036c631d88243 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"e762f541a090b2c3aaff1b9f","created_at":"2026-07-04T13:33:01.204Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-af757cf0875f4ee4","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.207Z","job_size_bytes":185,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-e762f541a090b2c3aaff1b9f: done: 0.563481 sec","job_status":"done","queue_duration_s":0.022543,"scheduling_latency_s":0.022584,"gitaly_calls":7,"gitaly_duration_s":0.175422,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.027055,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.003741,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001632,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001316,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.00229,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009188,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008888,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.105,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.105,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000770489,"exclusive_lock_hold_duration_s":0.055576896,"duration_s":0.563481,"completed_at":"2026-07-04T13:33:01.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.111255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1952vm-test-run-nixbot-gitlab> gitlab # [  206.852409] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.794Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0be026c896121c0f8b7a2143","created_at":"2026-07-04T13:33:01.476Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.478Z","job_size_bytes":46,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0be026c896121c0f8b7a2143: start","job_status":"start","queue_duration_s":0.31652,"scheduling_latency_s":0.316543}1953vm-test-run-nixbot-gitlab> gitlab # [  206.854213] systemd[1]: Started Nix Daemon.1954vm-test-run-nixbot-gitlab> gitlab # [  206.911236] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.922Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0be026c896121c0f8b7a2143","created_at":"2026-07-04T13:33:01.476Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.478Z","job_size_bytes":46,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0be026c896121c0f8b7a2143: done: 0.128232 sec","job_status":"done","queue_duration_s":0.31652,"scheduling_latency_s":0.316543,"redis_calls":9,"redis_duration_s":0.002695,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000284,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.128232,"completed_at":"2026-07-04T13:33:01.922Z","load_balancing_strategy":"primary","db_duration_s":0.001916,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1955vm-test-run-nixbot-gitlab> gitlab # [  206.915857] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:01.927Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2f8f699c5e5a9946f1da0a68","created_at":"2026-07-04T13:33:01.482Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.484Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2f8f699c5e5a9946f1da0a68: start","job_status":"start","queue_duration_s":0.443693,"scheduling_latency_s":0.443721}1956vm-test-run-nixbot-gitlab> gitlab # [  206.970799] nix-daemon[3393]: accepted connection from pid 3391, user nixbot1957vm-test-run-nixbot-gitlab> gitlab # [  206.974834] nix-daemon[3393]: accepted connection from pid 3390, user nixbot1958vm-test-run-nixbot-gitlab> gitlab # [  207.036811] gitaly[1387]: time="2026-07-04T13:33:02.048Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE diskcache=cdcce996-da46-4d65-8e44-15f70e29ca21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T19:33:01.952" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.952" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1959vm-test-run-nixbot-gitlab> gitlab # [  207.052808] gitaly[1387]: time="2026-07-04T13:33:02.064Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1643 command.oublock=0 command.real_time_ms=100 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T19:33:01.952" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:01.952" grpc.time_ms=96.384 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960vm-test-run-nixbot-gitlab> gitlab # [  207.071620] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.083Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2f8f699c5e5a9946f1da0a68","created_at":"2026-07-04T13:33:01.482Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.484Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2f8f699c5e5a9946f1da0a68: done: 0.155428 sec","job_status":"done","queue_duration_s":0.443693,"scheduling_latency_s":0.443721,"gitaly_calls":1,"gitaly_duration_s":0.114675,"redis_calls":12,"redis_duration_s":0.0059900000000000005,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000656,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.129873769,"duration_s":0.155428,"completed_at":"2026-07-04T13:33:02.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1961vm-test-run-nixbot-gitlab> gitlab # [  207.078631] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.084Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1c589614baf5dc7e5bb6cec4","created_at":"2026-07-04T13:33:01.575Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.583Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1c589614baf5dc7e5bb6cec4: start","job_status":"start","queue_duration_s":0.500799,"scheduling_latency_s":0.50082}1962vm-test-run-nixbot-gitlab> gitlab # [  207.113494] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.125Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1c589614baf5dc7e5bb6cec4","created_at":"2026-07-04T13:33:01.575Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.583Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1c589614baf5dc7e5bb6cec4: done: 0.040612 sec","job_status":"done","queue_duration_s":0.500799,"scheduling_latency_s":0.50082,"redis_calls":18,"redis_duration_s":0.003973,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002185,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345854,"duration_s":0.040612,"completed_at":"2026-07-04T13:33:02.125Z","load_balancing_strategy":"primary","db_duration_s":0.008165,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1963vm-test-run-nixbot-gitlab> gitlab # [  207.179226] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.126Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6358ebcf86cc965e41e25d9c","created_at":"2026-07-04T13:33:01.584Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.588Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6358ebcf86cc965e41e25d9c: start","job_status":"start","queue_duration_s":0.53795,"scheduling_latency_s":0.53797}1964vm-test-run-nixbot-gitlab> gitlab # [  207.270640] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.176Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6358ebcf86cc965e41e25d9c","created_at":"2026-07-04T13:33:01.584Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.588Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6358ebcf86cc965e41e25d9c: done: 0.049542 sec","job_status":"done","queue_duration_s":0.53795,"scheduling_latency_s":0.53797,"redis_calls":15,"redis_duration_s":0.003374,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001383,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344736,"duration_s":0.049542,"completed_at":"2026-07-04T13:33:02.176Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965vm-test-run-nixbot-gitlab> gitlab # [  207.344320] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:02.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1e0c9be0169b837955f7edb2","created_at":"2026-07-04T13:33:01.639Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd96f1dda13f6d6abc3277d0e3119adffa6679d01fad252f10d1b5b604c4cb9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.640Z","job_size_bytes":467,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1e0c9be0169b837955f7edb2: start","job_status":"start","queue_duration_s":0.536851,"scheduling_latency_s":0.536872}1966vm-test-run-nixbot-gitlab> gitlab # [  207.346205] gitaly[1387]: time="2026-07-04T13:33:02.196Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:32.195" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:02.195" grpc.time_ms=0.921 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967vm-test-run-nixbot-gitlab> gitlab # [  207.406731] gitaly[1387]: time="2026-07-04T13:33:02.418Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T13:33:12.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:02.410" grpc.time_ms=7.582 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968vm-test-run-nixbot-gitlab> gitlab # [  207.433946] gitaly[1387]: time="2026-07-04T13:33:02.445Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:32.044" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:02.445" grpc.time_ms=0.82 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969vm-test-run-nixbot-gitlab> gitlab # [  207.464875] gitaly[1387]: time="2026-07-04T13:33:02.476Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=58 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T13:33:32.072" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:02.473" grpc.time_ms=3.47 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1970vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'1971vm-test-run-nixbot-gitlab> gitlab # [  207.526416] gitaly[1387]: time="2026-07-04T13:33:02.538Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:12.124" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:02.524" grpc.time_ms=13.225 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972vm-test-run-nixbot-gitlab> gitlab # [  207.547809] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60842 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK1973vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.05 seconds)1974vm-test-run-nixbot-gitlab> []1975vm-test-run-nixbot-gitlab> gitlab # [  208.009440] gitaly[1387]: time="2026-07-04T13:33:03.021Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T13:33:32.600" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:03.001" grpc.time_ms=19.436 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976vm-test-run-nixbot-gitlab> gitlab # [  208.070694] gitaly[1387]: time="2026-07-04T13:33:03.082Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7C17H75WN3R7S7CRQRE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T13:33:33.072" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:03.072" grpc.time_ms=9.547 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977vm-test-run-nixbot-gitlab> gitlab # [  208.093117] sidekiq-cluster[2137]: {"severity":"WARN","time":"2026-07-04T13:33:03.104Z","class":"CreatePipelineWorker","correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"2167be41b58e6269099e2a2588e308eb35c22060","after":"fe2a76c9ff364b509bf59bd688e036c631d88243","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"fe2a76c9ff364b509bf59bd688e036c631d88243","ref":"refs/heads/master"},"retry":0}1978vm-test-run-nixbot-gitlab> gitlab # [  208.122863] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:03.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1e0c9be0169b837955f7edb2","created_at":"2026-07-04T13:33:01.639Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd96f1dda13f6d6abc3277d0e3119adffa6679d01fad252f10d1b5b604c4cb9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.640Z","job_size_bytes":467,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1e0c9be0169b837955f7edb2: done: 0.956158 sec","job_status":"done","queue_duration_s":0.536851,"scheduling_latency_s":0.536872,"gitaly_calls":7,"gitaly_duration_s":0.337638,"redis_calls":33,"redis_duration_s":0.011949,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001114,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003796,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.495122,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.956158,"completed_at":"2026-07-04T13:33:03.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1979vm-test-run-nixbot-gitlab> gitlab # [  208.135734] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:03.147Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"49a0243da61a1069793acb6c","created_at":"2026-07-04T13:33:01.684Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.690Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-49a0243da61a1069793acb6c: start","job_status":"start","queue_duration_s":1.456347,"scheduling_latency_s":1.45657}1980vm-test-run-nixbot-gitlab> gitlab # [  208.345246] gitaly[1387]: time="2026-07-04T13:33:03.357Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:33.355" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:03.356" grpc.time_ms=0.95 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981vm-test-run-nixbot-gitlab> gitlab # [  208.445750] gitaly[1387]: time="2026-07-04T13:33:03.456Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:33.054" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:03.455" grpc.time_ms=1.259 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'1983vm-test-run-nixbot-gitlab> gitlab # [  208.586823] gitaly[1387]: time="2026-07-04T13:33:03.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNA7T5X4XM7GAMZBYBQV1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:33.195" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:03.597" grpc.time_ms=1.355 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984vm-test-run-nixbot-gitlab> gitlab # [  208.593744] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK1985vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.07 seconds)1986vm-test-run-nixbot-gitlab> []1987vm-test-run-nixbot-gitlab> gitlab # [  208.651308] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:03.661Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"49a0243da61a1069793acb6c","created_at":"2026-07-04T13:33:01.684Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-3f22d08220154e18","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.690Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-49a0243da61a1069793acb6c: done: 0.514164 sec","job_status":"done","queue_duration_s":1.456347,"scheduling_latency_s":1.45657,"gitaly_calls":3,"gitaly_duration_s":0.058252,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.055796,"redis_read_bytes":209,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.007964,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.006428,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.014191,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000584,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.008839,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001151,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00315,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005525,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.215,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.02,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.11,"db_ci_duration_s":0.105,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174804,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.426853412,"duration_s":0.514164,"completed_at":"2026-07-04T13:33:03.661Z","load_balancing_strategy":"primary","db_duration_s":0.226567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1988vm-test-run-nixbot-gitlab> gitlab # [  208.738253] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:03.749Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"35fee20b-f661-4cd4-a713-10d65f339631\"}"],"class":"WebHookWorker","jid":"8a5bae0b5344c4f2a9e7ed37","created_at":"2026-07-04T13:33:01.731Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:273d26c673aeb08ba1748721cb5d8d51f293686ed6f5d524b9dc1d551f05b074","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.737Z","job_size_bytes":1749,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8a5bae0b5344c4f2a9e7ed37: start","job_status":"start","queue_duration_s":2.011636,"scheduling_latency_s":2.011696}1989vm-test-run-nixbot-gitlab> gitlab # [  208.890114] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60860 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1990vm-test-run-nixbot-gitlab> gitlab # [  208.919324] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:03.930Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"35fee20b-f661-4cd4-a713-10d65f339631\"}"],"class":"WebHookWorker","jid":"8a5bae0b5344c4f2a9e7ed37","created_at":"2026-07-04T13:33:01.731Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a57f6200c2d04dd3","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:273d26c673aeb08ba1748721cb5d8d51f293686ed6f5d524b9dc1d551f05b074","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:01.737Z","job_size_bytes":1749,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8a5bae0b5344c4f2a9e7ed37: done: 0.181444 sec","job_status":"done","queue_duration_s":2.011636,"scheduling_latency_s":2.011696,"redis_calls":19,"redis_duration_s":0.009254,"redis_read_bytes":32,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002562,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000534,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0039,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002258,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.017646656000010807,"cpu_s":0.141597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000569625,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.181444,"completed_at":"2026-07-04T13:33:03.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991vm-test-run-nixbot-gitlab> gitlab # [  209.025643] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a0ad8ea1d3014c842fe944a4","created_at":"2026-07-04T13:33:02.980Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89298281913aa1609b3a2705c6dd0799","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:02.994Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a0ad8ea1d3014c842fe944a4: start","job_status":"start","queue_duration_s":1.042445,"scheduling_latency_s":1.042483}1992vm-test-run-nixbot-gitlab> gitlab # [  209.064670] gitaly[1387]: time="2026-07-04T13:33:04.076Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAAJFC1E0H51XQ4DN4ATY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:04.068" grpc.time_ms=8.005 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc1993vm-test-run-nixbot-gitlab> gitlab # [  209.066812] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 1251994vm-test-run-nixbot-gitlab> gitlab # [  209.108559] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a0ad8ea1d3014c842fe944a4","created_at":"2026-07-04T13:33:02.980Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89298281913aa1609b3a2705c6dd0799","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:02.994Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a0ad8ea1d3014c842fe944a4: done: 0.082976 sec","job_status":"done","queue_duration_s":1.042445,"scheduling_latency_s":1.042483,"redis_calls":9,"redis_duration_s":0.0028640000000000002,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001535,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000977499,"exclusive_lock_hold_duration_s":0.010914236,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.082976,"completed_at":"2026-07-04T13:33:04.120Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1995vm-test-run-nixbot-gitlab> gitlab # [  209.110529] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"04f2f6785c99180a51aa57c8","created_at":"2026-07-04T13:33:03.005Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49346f572ad31344bc74bb5efa4e8423","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.030Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-04f2f6785c99180a51aa57c8: start","job_status":"start","queue_duration_s":1.091004,"scheduling_latency_s":1.091025}1996vm-test-run-nixbot-gitlab> gitlab # [  209.182362] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"04f2f6785c99180a51aa57c8","created_at":"2026-07-04T13:33:03.005Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49346f572ad31344bc74bb5efa4e8423","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.030Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-04f2f6785c99180a51aa57c8: done: 0.025127 sec","job_status":"done","queue_duration_s":1.091004,"scheduling_latency_s":1.091025,"redis_calls":12,"redis_duration_s":0.0030340000000000002,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000503695,"exclusive_lock_hold_duration_s":0.0062298430000000005,"duration_s":0.025127,"completed_at":"2026-07-04T13:33:04.147Z","load_balancing_strategy":"primary","db_duration_s":0.002471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997vm-test-run-nixbot-gitlab> gitlab # [  209.193142] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"9a258adcda01b873e2ee569f","created_at":"2026-07-04T13:33:03.041Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.050Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-9a258adcda01b873e2ee569f: start","job_status":"start","queue_duration_s":1.097981,"scheduling_latency_s":1.098002}1998vm-test-run-nixbot-gitlab> gitlab # [  209.278541] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"9a258adcda01b873e2ee569f","created_at":"2026-07-04T13:33:03.041Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.050Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-9a258adcda01b873e2ee569f: done: 0.036165 sec","job_status":"done","queue_duration_s":1.097981,"scheduling_latency_s":1.098002,"redis_calls":28,"redis_duration_s":0.00657,"redis_read_bytes":36,"redis_write_bytes":10593,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000908,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000396,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8305,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.003857,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001409,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000351721,"duration_s":0.036165,"completed_at":"2026-07-04T13:33:04.184Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999vm-test-run-nixbot-gitlab> gitlab # [  209.286136] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7ec0baa39857fb54cc35c277","created_at":"2026-07-04T13:33:03.065Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b0df26b61a258306b8bc9b2a57b3950","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T13:33:03.067Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7ec0baa39857fb54cc35c277: start","job_status":"start","queue_duration_s":1.118609,"scheduling_latency_s":1.118629}2000vm-test-run-nixbot-gitlab> gitlab # [  209.368707] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.198Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2001vm-test-run-nixbot-gitlab> gitlab # [  209.370493] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.203Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7ec0baa39857fb54cc35c277","created_at":"2026-07-04T13:33:03.065Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b0df26b61a258306b8bc9b2a57b3950","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T13:33:03.067Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7ec0baa39857fb54cc35c277: done: 0.017665 sec","job_status":"done","queue_duration_s":1.118609,"scheduling_latency_s":1.118629,"redis_calls":7,"redis_duration_s":0.006013,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017665,"completed_at":"2026-07-04T13:33:04.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2002vm-test-run-nixbot-gitlab> gitlab # [  209.377296] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a765edcf9527d12b03437b1a","created_at":"2026-07-04T13:33:03.079Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10b97a6bed3b552496426bd2ba0a2bef","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.084Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a765edcf9527d12b03437b1a: start","job_status":"start","queue_duration_s":1.12045,"scheduling_latency_s":1.120481}2003vm-test-run-nixbot-gitlab> gitlab # [  209.401981] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a765edcf9527d12b03437b1a","created_at":"2026-07-04T13:33:03.079Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10b97a6bed3b552496426bd2ba0a2bef","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.084Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a765edcf9527d12b03437b1a: done: 0.016711 sec","job_status":"done","queue_duration_s":1.12045,"scheduling_latency_s":1.120481,"redis_calls":9,"redis_duration_s":0.0021019999999999997,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000500623,"exclusive_lock_hold_duration_s":0.003231696,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016711,"completed_at":"2026-07-04T13:33:04.221Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004vm-test-run-nixbot-gitlab> gitlab # [  209.517473] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f2923bd396ba09437b15c390","created_at":"2026-07-04T13:33:03.094Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d2cecb4b5ad33636f3308161fd6dd56","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.098Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f2923bd396ba09437b15c390: start","job_status":"start","queue_duration_s":1.124419,"scheduling_latency_s":1.124438}2005vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'2006vm-test-run-nixbot-gitlab> gitlab # [  209.519461] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.241Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f2923bd396ba09437b15c390","created_at":"2026-07-04T13:33:03.094Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d2cecb4b5ad33636f3308161fd6dd56","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.098Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f2923bd396ba09437b15c390: done: 0.017989 sec","job_status":"done","queue_duration_s":1.124419,"scheduling_latency_s":1.124438,"redis_calls":7,"redis_duration_s":0.001929,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017989,"completed_at":"2026-07-04T13:33:04.241Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007vm-test-run-nixbot-gitlab> gitlab # [  209.618319] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0fc584cb9e16d8ee096fc0a0","created_at":"2026-07-04T13:33:03.123Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7cf7bc7b4a32c91878a27f9432821e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.129Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0fc584cb9e16d8ee096fc0a0: start","job_status":"start","queue_duration_s":1.112923,"scheduling_latency_s":1.112946}2008vm-test-run-nixbot-gitlab> gitlab # [  209.619918] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.250Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2009vm-test-run-nixbot-gitlab> gitlab # [  209.621439] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0fc584cb9e16d8ee096fc0a0","created_at":"2026-07-04T13:33:03.123Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7cf7bc7b4a32c91878a27f9432821e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.129Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0fc584cb9e16d8ee096fc0a0: done: 0.012007 sec","job_status":"done","queue_duration_s":1.112923,"scheduling_latency_s":1.112946,"redis_calls":7,"redis_duration_s":0.0016450000000000002,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012007,"completed_at":"2026-07-04T13:33:04.254Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.11 seconds)2011vm-test-run-nixbot-gitlab> []2012vm-test-run-nixbot-gitlab> gitlab # [  209.695759] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"46d03a285690996d61c88738","created_at":"2026-07-04T13:33:03.140Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2901d866fa5b5a4a4c1c28bed7d81d2a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.146Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-46d03a285690996d61c88738: start","job_status":"start","queue_duration_s":1.119624,"scheduling_latency_s":1.119646}2013vm-test-run-nixbot-gitlab> gitlab # [  209.697055] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"46d03a285690996d61c88738","created_at":"2026-07-04T13:33:03.140Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2901d866fa5b5a4a4c1c28bed7d81d2a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.146Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-46d03a285690996d61c88738: done: 0.014183 sec","job_status":"done","queue_duration_s":1.119624,"scheduling_latency_s":1.119646,"redis_calls":7,"redis_duration_s":0.0018349999999999998,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014183,"completed_at":"2026-07-04T13:33:04.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014vm-test-run-nixbot-gitlab> gitlab # [  209.780343] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3601a627a96c4321e711d35d","created_at":"2026-07-04T13:33:03.173Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"878dcb0a786883855e76f816e4dc8096","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.180Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3601a627a96c4321e711d35d: start","job_status":"start","queue_duration_s":1.101394,"scheduling_latency_s":1.101414}2015vm-test-run-nixbot-gitlab> gitlab # [  209.781818] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.289Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2016vm-test-run-nixbot-gitlab> gitlab # [  209.782958] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3601a627a96c4321e711d35d","created_at":"2026-07-04T13:33:03.173Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"878dcb0a786883855e76f816e4dc8096","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.180Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3601a627a96c4321e711d35d: done: 0.01201 sec","job_status":"done","queue_duration_s":1.101394,"scheduling_latency_s":1.101414,"redis_calls":7,"redis_duration_s":0.001573,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01201,"completed_at":"2026-07-04T13:33:04.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017vm-test-run-nixbot-gitlab> gitlab # [  209.860949] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7daf1f2796b55789e4f01ad0","created_at":"2026-07-04T13:33:03.203Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe0e73d2e8490b7f5b18b40e05f2a7d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.207Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7daf1f2796b55789e4f01ad0: start","job_status":"start","queue_duration_s":1.08811,"scheduling_latency_s":1.08813}2018vm-test-run-nixbot-gitlab> gitlab # [  209.862069] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.302Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2019vm-test-run-nixbot-gitlab> gitlab # [  209.862865] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.307Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7daf1f2796b55789e4f01ad0","created_at":"2026-07-04T13:33:03.203Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe0e73d2e8490b7f5b18b40e05f2a7d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.207Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7daf1f2796b55789e4f01ad0: done: 0.011857 sec","job_status":"done","queue_duration_s":1.08811,"scheduling_latency_s":1.08813,"redis_calls":7,"redis_duration_s":0.0018509999999999998,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011857,"completed_at":"2026-07-04T13:33:04.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020vm-test-run-nixbot-gitlab> gitlab # [  209.938164] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ea6ce84d571743f07377e237","created_at":"2026-07-04T13:33:03.213Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.215Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ea6ce84d571743f07377e237: start","job_status":"start","queue_duration_s":1.092732,"scheduling_latency_s":1.092752}2021vm-test-run-nixbot-gitlab> gitlab # [  210.013501] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ea6ce84d571743f07377e237","created_at":"2026-07-04T13:33:03.213Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.215Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ea6ce84d571743f07377e237: done: 0.080156 sec","job_status":"done","queue_duration_s":1.092732,"scheduling_latency_s":1.092752,"redis_calls":60,"redis_duration_s":0.014911,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000635,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013461,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080156,"completed_at":"2026-07-04T13:33:04.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022vm-test-run-nixbot-gitlab> gitlab # [  210.015278] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2587c9306e3093a3e2b044a9","created_at":"2026-07-04T13:33:03.233Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26f91e236eabedb874b75613a308c647","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.235Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2587c9306e3093a3e2b044a9: start","job_status":"start","queue_duration_s":1.154442,"scheduling_latency_s":1.154468}2023vm-test-run-nixbot-gitlab> gitlab # [  210.034951] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2587c9306e3093a3e2b044a9","created_at":"2026-07-04T13:33:03.233Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26f91e236eabedb874b75613a308c647","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.235Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2587c9306e3093a3e2b044a9: done: 0.018384 sec","job_status":"done","queue_duration_s":1.154442,"scheduling_latency_s":1.154468,"redis_calls":9,"redis_duration_s":0.0020540000000000003,"redis_read_bytes":7,"redis_write_bytes":1108,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":277,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018384,"completed_at":"2026-07-04T13:33:04.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024vm-test-run-nixbot-gitlab> gitlab # [  210.111975] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3ba0e6f5e99d9dcaa83754dd","created_at":"2026-07-04T13:33:03.259Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"664fcb7685a3876ca720e6fe3b8cef60","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.263Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3ba0e6f5e99d9dcaa83754dd: start","job_status":"start","queue_duration_s":1.147081,"scheduling_latency_s":1.147104}2025vm-test-run-nixbot-gitlab> gitlab # [  210.124675] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3ba0e6f5e99d9dcaa83754dd","created_at":"2026-07-04T13:33:03.259Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"664fcb7685a3876ca720e6fe3b8cef60","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.263Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3ba0e6f5e99d9dcaa83754dd: done: 0.019701 sec","job_status":"done","queue_duration_s":1.147081,"scheduling_latency_s":1.147104,"redis_calls":7,"redis_duration_s":0.0016870000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019701,"completed_at":"2026-07-04T13:33:04.430Z","load_balancing_strategy":"primary","db_duration_s":0.007046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026vm-test-run-nixbot-gitlab> gitlab # [  210.199073] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3e72e27c96f180b174136ffc","created_at":"2026-07-04T13:33:03.285Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a245931c653ccaa523a69997be1ed5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.289Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3e72e27c96f180b174136ffc: start","job_status":"start","queue_duration_s":1.141865,"scheduling_latency_s":1.141887}2027vm-test-run-nixbot-gitlab> gitlab # [  210.277067] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3e72e27c96f180b174136ffc","created_at":"2026-07-04T13:33:03.285Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a245931c653ccaa523a69997be1ed5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.289Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3e72e27c96f180b174136ffc: done: 0.014074 sec","job_status":"done","queue_duration_s":1.141865,"scheduling_latency_s":1.141887,"redis_calls":7,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014074,"completed_at":"2026-07-04T13:33:04.445Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028vm-test-run-nixbot-gitlab> gitlab # [  210.279235] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dbd50f708565159fd36da0be","created_at":"2026-07-04T13:33:03.303Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1a35004292a58555145c2fa3ba209d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T13:33:03.307Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dbd50f708565159fd36da0be: start","job_status":"start","queue_duration_s":1.139219,"scheduling_latency_s":1.13924}2029vm-test-run-nixbot-gitlab> gitlab # [  210.349195] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.454Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2030vm-test-run-nixbot-gitlab> gitlab # [  210.350724] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dbd50f708565159fd36da0be","created_at":"2026-07-04T13:33:03.303Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1a35004292a58555145c2fa3ba209d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T13:33:03.307Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dbd50f708565159fd36da0be: done: 0.012015 sec","job_status":"done","queue_duration_s":1.139219,"scheduling_latency_s":1.13924,"redis_calls":7,"redis_duration_s":0.001627,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012015,"completed_at":"2026-07-04T13:33:04.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031vm-test-run-nixbot-gitlab> gitlab # [  210.432504] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8cea79c7c37fbd463d8a241f","created_at":"2026-07-04T13:33:03.303Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-38e9a999ff8248f9","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.312Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8cea79c7c37fbd463d8a241f: start","job_status":"start","queue_duration_s":1.148096,"scheduling_latency_s":1.148117}2032vm-test-run-nixbot-gitlab> gitlab # [  210.433892] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.490Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8cea79c7c37fbd463d8a241f","created_at":"2026-07-04T13:33:03.303Z","trace_propagation_headers":{"sentry-trace":"9e3be37cf9384c739c3275ec4278ad40-38e9a999ff8248f9","baggage":"sentry-trace_id=9e3be37cf9384c739c3275ec4278ad40,sentry-release=Unknown"},"correlation_id":"01KWPNA7T5X4XM7GAMZBYBQV1T","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.312Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8cea79c7c37fbd463d8a241f: done: 0.029879 sec","job_status":"done","queue_duration_s":1.148096,"scheduling_latency_s":1.148117,"redis_calls":10,"redis_duration_s":0.002381,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000485,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348647,"duration_s":0.029879,"completed_at":"2026-07-04T13:33:04.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033vm-test-run-nixbot-gitlab> gitlab # [  210.522616] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"aafc6721b29b044d9f995972","created_at":"2026-07-04T13:33:03.320Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84c47e4c238bfba395baf05380b692d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.323Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-aafc6721b29b044d9f995972: start","job_status":"start","queue_duration_s":1.168342,"scheduling_latency_s":1.168363}2034vm-test-run-nixbot-gitlab> gitlab # [  210.525865] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.499Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2035vm-test-run-nixbot-gitlab> gitlab # [  210.527108] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"aafc6721b29b044d9f995972","created_at":"2026-07-04T13:33:03.320Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84c47e4c238bfba395baf05380b692d8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.323Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-aafc6721b29b044d9f995972: done: 0.012006 sec","job_status":"done","queue_duration_s":1.168342,"scheduling_latency_s":1.168363,"redis_calls":7,"redis_duration_s":0.001559,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012006,"completed_at":"2026-07-04T13:33:04.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2036vm-test-run-nixbot-gitlab> gitlab # [  210.607312] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e6a52c53d911bf1ec8a0bd27","created_at":"2026-07-04T13:33:03.356Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cccfc800dee04e6dad94ae8174ee61e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T13:33:03.361Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e6a52c53d911bf1ec8a0bd27: start","job_status":"start","queue_duration_s":1.143895,"scheduling_latency_s":1.143914}2037vm-test-run-nixbot-gitlab> gitlab # [  210.608833] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e6a52c53d911bf1ec8a0bd27","created_at":"2026-07-04T13:33:03.356Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cccfc800dee04e6dad94ae8174ee61e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T13:33:03.361Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e6a52c53d911bf1ec8a0bd27: done: 0.017908 sec","job_status":"done","queue_duration_s":1.143895,"scheduling_latency_s":1.143914,"redis_calls":7,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017908,"completed_at":"2026-07-04T13:33:04.522Z","load_balancing_strategy":"primary","db_duration_s":0.002179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038vm-test-run-nixbot-gitlab> gitlab # [  210.687199] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8b2ec467ced364fb66843c6d","created_at":"2026-07-04T13:33:03.398Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17226aaf5715fe01957dfb22dede36ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.401Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8b2ec467ced364fb66843c6d: start","job_status":"start","queue_duration_s":1.123026,"scheduling_latency_s":1.123087}2039vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'2040vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.03 seconds)2041vm-test-run-nixbot-gitlab> []2042vm-test-run-nixbot-gitlab> gitlab # [  210.688767] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8b2ec467ced364fb66843c6d","created_at":"2026-07-04T13:33:03.398Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17226aaf5715fe01957dfb22dede36ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.401Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8b2ec467ced364fb66843c6d: done: 0.149924 sec","job_status":"done","queue_duration_s":1.123026,"scheduling_latency_s":1.123087,"redis_calls":9,"redis_duration_s":0.002349,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":123,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":123,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000452851,"exclusive_lock_hold_duration_s":0.135453249,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.149924,"completed_at":"2026-07-04T13:33:04.674Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043vm-test-run-nixbot-gitlab> gitlab # [  210.772246] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"64a5a16be10e6ded9e98e4d7","created_at":"2026-07-04T13:33:03.423Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.431Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-64a5a16be10e6ded9e98e4d7: start","job_status":"start","queue_duration_s":1.244988,"scheduling_latency_s":1.24501}2044vm-test-run-nixbot-gitlab> gitlab # [  210.773743] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"64a5a16be10e6ded9e98e4d7","created_at":"2026-07-04T13:33:03.423Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.431Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-64a5a16be10e6ded9e98e4d7: done: 0.059338 sec","job_status":"done","queue_duration_s":1.244988,"scheduling_latency_s":1.24501,"redis_calls":14,"redis_duration_s":0.0039440000000000005,"redis_read_bytes":16,"redis_write_bytes":4022,"redis_queues_calls":2,"redis_queues_duration_s":0.000295,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002906,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059338,"completed_at":"2026-07-04T13:33:04.735Z","load_balancing_strategy":"primary","db_duration_s":0.035231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045vm-test-run-nixbot-gitlab> gitlab # [  210.852131] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60874 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK2046vm-test-run-nixbot-gitlab> gitlab # [  210.852901] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60890 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK2047vm-test-run-nixbot-gitlab> gitlab # [  210.854179] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:04.736Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f082c750b5181fb51f53bcd8","created_at":"2026-07-04T13:33:03.446Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4444317693dc98afdd95e9883a2dad97","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.452Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f082c750b5181fb51f53bcd8: start","job_status":"start","queue_duration_s":1.284453,"scheduling_latency_s":1.284476}2048vm-test-run-nixbot-gitlab> gitlab # [  210.855239] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f082c750b5181fb51f53bcd8","created_at":"2026-07-04T13:33:03.446Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4444317693dc98afdd95e9883a2dad97","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.452Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f082c750b5181fb51f53bcd8: done: 0.398736 sec","job_status":"done","queue_duration_s":1.284453,"scheduling_latency_s":1.284476,"redis_calls":571,"redis_duration_s":0.125189,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.123975,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.296824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.398736,"completed_at":"2026-07-04T13:33:05.135Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049vm-test-run-nixbot-gitlab> gitlab # [  210.933656] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e68ad387c05e8823bed6283c","created_at":"2026-07-04T13:33:03.520Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f03c4509b231b625a1bd8c021d361b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.528Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e68ad387c05e8823bed6283c: start","job_status":"start","queue_duration_s":1.608548,"scheduling_latency_s":1.608574}2050vm-test-run-nixbot-gitlab> gitlab # [  210.935112] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e68ad387c05e8823bed6283c","created_at":"2026-07-04T13:33:03.520Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f03c4509b231b625a1bd8c021d361b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.528Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e68ad387c05e8823bed6283c: done: 0.026126 sec","job_status":"done","queue_duration_s":1.608548,"scheduling_latency_s":1.608574,"redis_calls":9,"redis_duration_s":0.002637,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000481905,"exclusive_lock_hold_duration_s":0.012354643,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.026126,"completed_at":"2026-07-04T13:33:05.163Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2051vm-test-run-nixbot-gitlab> gitlab # [  211.033370] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6d8be7bcffaa38f366132615","created_at":"2026-07-04T13:33:03.566Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"971072eacf722d44d7bd0e91e1e2ddd1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.578Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6d8be7bcffaa38f366132615: start","job_status":"start","queue_duration_s":1.585623,"scheduling_latency_s":1.585644}2052vm-test-run-nixbot-gitlab> gitlab # [  211.034901] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6d8be7bcffaa38f366132615","created_at":"2026-07-04T13:33:03.566Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"971072eacf722d44d7bd0e91e1e2ddd1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.578Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6d8be7bcffaa38f366132615: done: 0.016728 sec","job_status":"done","queue_duration_s":1.585623,"scheduling_latency_s":1.585644,"redis_calls":10,"redis_duration_s":0.002386,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003301537,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.016728,"completed_at":"2026-07-04T13:33:05.181Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001939,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2053vm-test-run-nixbot-gitlab> gitlab # [  211.150046] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"da93fa35f6fee9f568c618b1","created_at":"2026-07-04T13:33:03.598Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fef2f52b2c8c5d9eb321340d2722637","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.641Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-da93fa35f6fee9f568c618b1: start","job_status":"start","queue_duration_s":1.541414,"scheduling_latency_s":1.541434}2054vm-test-run-nixbot-gitlab> gitlab # [  211.151082] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"da93fa35f6fee9f568c618b1","created_at":"2026-07-04T13:33:03.598Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-e8d7cc69edfd41ed","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6fef2f52b2c8c5d9eb321340d2722637","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.641Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-da93fa35f6fee9f568c618b1: done: 0.01411 sec","job_status":"done","queue_duration_s":1.541414,"scheduling_latency_s":1.541434,"redis_calls":9,"redis_duration_s":0.0022199999999999998,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000455365,"exclusive_lock_hold_duration_s":0.001903594,"duration_s":0.01411,"completed_at":"2026-07-04T13:33:05.197Z","load_balancing_strategy":"primary","db_duration_s":0.000727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055vm-test-run-nixbot-gitlab> gitlab # [  211.229805] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.198Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8a5bae0b5344c4f2a9e7ed37"],"class":"WebHooks::LogExecutionWorker","jid":"a86c75fc34b5209f4b7b5626","created_at":"2026-07-04T13:33:03.909Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a9c2568ec9314010","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5104bebd5f2d4ba47581b66da7f619eba7d40483f1156a9de3b56ebc54459120","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.920Z","job_size_bytes":2507,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a86c75fc34b5209f4b7b5626: start","job_status":"start","queue_duration_s":1.277711,"scheduling_latency_s":1.277731}2056vm-test-run-nixbot-gitlab> gitlab # [  211.231113] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.301Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8a5bae0b5344c4f2a9e7ed37"],"class":"WebHooks::LogExecutionWorker","jid":"a86c75fc34b5209f4b7b5626","created_at":"2026-07-04T13:33:03.909Z","trace_propagation_headers":{"sentry-trace":"5dec1b6e1f1545ffb1de75d98ee83a88-a9c2568ec9314010","baggage":"sentry-trace_id=5dec1b6e1f1545ffb1de75d98ee83a88,sentry-release=Unknown"},"correlation_id":"01KWPNA7C17H75WN3R7S7CRQRE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5104bebd5f2d4ba47581b66da7f619eba7d40483f1156a9de3b56ebc54459120","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:03.920Z","job_size_bytes":2507,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a86c75fc34b5209f4b7b5626: done: 0.103091 sec","job_status":"done","queue_duration_s":1.277711,"scheduling_latency_s":1.277731,"redis_calls":10,"redis_duration_s":0.002588,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361499,"duration_s":0.103091,"completed_at":"2026-07-04T13:33:05.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057vm-test-run-nixbot-gitlab> gitlab # [  211.270937] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"bbf5f0fb7790c08c1f831838","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-bbf5f0fb7790c08c1f831838: start","job_status":"start","queue_duration_s":1.1256,"scheduling_latency_s":1.125622}2058vm-test-run-nixbot-gitlab> gitlab # [  211.349293] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"bbf5f0fb7790c08c1f831838","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-bbf5f0fb7790c08c1f831838: done: 0.020433 sec","job_status":"done","queue_duration_s":1.1256,"scheduling_latency_s":1.125622,"redis_calls":16,"redis_duration_s":0.0036880000000000003,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001124,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002144,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000383009,"duration_s":0.020433,"completed_at":"2026-07-04T13:33:05.323Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059vm-test-run-nixbot-gitlab> gitlab # [  211.355712] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b0847e367a3a5e69c700f662","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b0847e367a3a5e69c700f662: start","job_status":"start","queue_duration_s":1.147351,"scheduling_latency_s":1.147371}2060vm-test-run-nixbot-gitlab> gitlab # [  211.424996] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b0847e367a3a5e69c700f662","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b0847e367a3a5e69c700f662: done: 0.015076 sec","job_status":"done","queue_duration_s":1.147351,"scheduling_latency_s":1.147371,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015076,"completed_at":"2026-07-04T13:33:05.339Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2061vm-test-run-nixbot-gitlab> gitlab # [  211.437685] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e99be52958a3c7e13e59bfe3","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e99be52958a3c7e13e59bfe3: start","job_status":"start","queue_duration_s":1.163817,"scheduling_latency_s":1.163838}2062vm-test-run-nixbot-gitlab> gitlab # [  211.509895] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e99be52958a3c7e13e59bfe3","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e99be52958a3c7e13e59bfe3: done: 0.01528 sec","job_status":"done","queue_duration_s":1.163817,"scheduling_latency_s":1.163838,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01528,"completed_at":"2026-07-04T13:33:05.356Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063vm-test-run-nixbot-gitlab> gitlab # [  211.519407] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b4687478c147cd9e3761c623","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b4687478c147cd9e3761c623: start","job_status":"start","queue_duration_s":1.180433,"scheduling_latency_s":1.180453}2064vm-test-run-nixbot-gitlab> gitlab # [  211.596834] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b4687478c147cd9e3761c623","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b4687478c147cd9e3761c623: done: 0.016782 sec","job_status":"done","queue_duration_s":1.180433,"scheduling_latency_s":1.180453,"redis_calls":13,"redis_duration_s":0.002955,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016782,"completed_at":"2026-07-04T13:33:05.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065vm-test-run-nixbot-gitlab> gitlab # [  211.602442] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6282a37e8c7baf851e1894cb","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6282a37e8c7baf851e1894cb: start","job_status":"start","queue_duration_s":1.198501,"scheduling_latency_s":1.198523}2066vm-test-run-nixbot-gitlab> gitlab # [  211.690187] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6282a37e8c7baf851e1894cb","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6282a37e8c7baf851e1894cb: done: 0.014748 sec","job_status":"done","queue_duration_s":1.198501,"scheduling_latency_s":1.198523,"redis_calls":13,"redis_duration_s":0.003103,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014748,"completed_at":"2026-07-04T13:33:05.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2067vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'2068vm-test-run-nixbot-gitlab> gitlab # [  211.692300] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"941c481d3304744bf97761cb","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-941c481d3304744bf97761cb: start","job_status":"start","queue_duration_s":1.21452,"scheduling_latency_s":1.214541}2069vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.04 seconds)2070vm-test-run-nixbot-gitlab> []2071vm-test-run-nixbot-gitlab> gitlab # [  211.707934] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"941c481d3304744bf97761cb","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-941c481d3304744bf97761cb: done: 0.014656 sec","job_status":"done","queue_duration_s":1.21452,"scheduling_latency_s":1.214541,"redis_calls":13,"redis_duration_s":0.00291,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014656,"completed_at":"2026-07-04T13:33:05.406Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072vm-test-run-nixbot-gitlab> gitlab # [  211.783358] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4c057ffdf636d693adcf1f2b","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4c057ffdf636d693adcf1f2b: start","job_status":"start","queue_duration_s":1.230654,"scheduling_latency_s":1.230707}2073vm-test-run-nixbot-gitlab> gitlab # [  211.790865] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4c057ffdf636d693adcf1f2b","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4c057ffdf636d693adcf1f2b: done: 0.016182 sec","job_status":"done","queue_duration_s":1.230654,"scheduling_latency_s":1.230707,"redis_calls":13,"redis_duration_s":0.003194,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016182,"completed_at":"2026-07-04T13:33:05.424Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074vm-test-run-nixbot-gitlab> gitlab # [  211.850969] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f83180551f038e3ea821531c","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f83180551f038e3ea821531c: start","job_status":"start","queue_duration_s":1.248118,"scheduling_latency_s":1.248139}2075vm-test-run-nixbot-gitlab> gitlab # [  211.923344] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"f83180551f038e3ea821531c","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-f83180551f038e3ea821531c: done: 0.015884 sec","job_status":"done","queue_duration_s":1.248118,"scheduling_latency_s":1.248139,"redis_calls":13,"redis_duration_s":0.00379,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015884,"completed_at":"2026-07-04T13:33:05.441Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076vm-test-run-nixbot-gitlab> gitlab # [  211.932138] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c8328e0ed461b5845ff91aa3","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c8328e0ed461b5845ff91aa3: start","job_status":"start","queue_duration_s":1.265435,"scheduling_latency_s":1.265455}2077vm-test-run-nixbot-gitlab> gitlab # [  211.998083] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c8328e0ed461b5845ff91aa3","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c8328e0ed461b5845ff91aa3: done: 0.01696 sec","job_status":"done","queue_duration_s":1.265435,"scheduling_latency_s":1.265455,"redis_calls":13,"redis_duration_s":0.003225,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01696,"completed_at":"2026-07-04T13:33:05.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078vm-test-run-nixbot-gitlab> gitlab # [  212.012447] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c629479a36d2015f013db146","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c629479a36d2015f013db146: start","job_status":"start","queue_duration_s":1.283776,"scheduling_latency_s":1.283797}2079vm-test-run-nixbot-gitlab> gitlab # [  212.079187] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c629479a36d2015f013db146","created_at":"2026-07-04T13:33:04.162Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b09b87cb92ad4577","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"3eeddadb73423e277f9136121dbcadae","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.177Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c629479a36d2015f013db146: done: 0.015006 sec","job_status":"done","queue_duration_s":1.283776,"scheduling_latency_s":1.283797,"redis_calls":13,"redis_duration_s":0.003169,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015006,"completed_at":"2026-07-04T13:33:05.476Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080vm-test-run-nixbot-gitlab> gitlab # [  212.093746] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea3e0b7a0b12324dbb1c0ae","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea3e0b7a0b12324dbb1c0ae: start","job_status":"start","queue_duration_s":1.09639,"scheduling_latency_s":1.096419}2081vm-test-run-nixbot-gitlab> gitlab # [  212.171885] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea3e0b7a0b12324dbb1c0ae","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea3e0b7a0b12324dbb1c0ae: done: 0.015393 sec","job_status":"done","queue_duration_s":1.09639,"scheduling_latency_s":1.096419,"redis_calls":13,"redis_duration_s":0.003088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015393,"completed_at":"2026-07-04T13:33:05.493Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082vm-test-run-nixbot-gitlab> gitlab # [  212.175131] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4434d1f7ee36c95a3f975827","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4434d1f7ee36c95a3f975827: start","job_status":"start","queue_duration_s":1.113151,"scheduling_latency_s":1.113172}2083vm-test-run-nixbot-gitlab> gitlab # [  212.194804] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4434d1f7ee36c95a3f975827","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4434d1f7ee36c95a3f975827: done: 0.015448 sec","job_status":"done","queue_duration_s":1.113151,"scheduling_latency_s":1.113172,"redis_calls":13,"redis_duration_s":0.003112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015448,"completed_at":"2026-07-04T13:33:05.510Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084vm-test-run-nixbot-gitlab> gitlab # [  212.305805] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaf76c836cd4ef10592baf47","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaf76c836cd4ef10592baf47: start","job_status":"start","queue_duration_s":1.129918,"scheduling_latency_s":1.129939}2085vm-test-run-nixbot-gitlab> gitlab # [  212.307722] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eaf76c836cd4ef10592baf47","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eaf76c836cd4ef10592baf47: done: 0.015552 sec","job_status":"done","queue_duration_s":1.129918,"scheduling_latency_s":1.129939,"redis_calls":13,"redis_duration_s":0.0030930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015552,"completed_at":"2026-07-04T13:33:05.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086vm-test-run-nixbot-gitlab> gitlab # [  212.383372] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967409c22415f3e89ed429e0","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967409c22415f3e89ed429e0: start","job_status":"start","queue_duration_s":1.147036,"scheduling_latency_s":1.147056}2087vm-test-run-nixbot-gitlab> gitlab # [  212.384892] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967409c22415f3e89ed429e0","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967409c22415f3e89ed429e0: done: 0.015507 sec","job_status":"done","queue_duration_s":1.147036,"scheduling_latency_s":1.147056,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015507,"completed_at":"2026-07-04T13:33:05.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088vm-test-run-nixbot-gitlab> gitlab # [  212.418084] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba28d4bf31fce5d4300cb8d2","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba28d4bf31fce5d4300cb8d2: start","job_status":"start","queue_duration_s":1.164061,"scheduling_latency_s":1.164082}2089vm-test-run-nixbot-gitlab> gitlab # [  212.437721] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba28d4bf31fce5d4300cb8d2","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba28d4bf31fce5d4300cb8d2: done: 0.039039 sec","job_status":"done","queue_duration_s":1.164061,"scheduling_latency_s":1.164082,"redis_calls":13,"redis_duration_s":0.003147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039039,"completed_at":"2026-07-04T13:33:05.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090vm-test-run-nixbot-gitlab> gitlab # [  212.519533] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb20d11b1ed6f446c91ae46c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb20d11b1ed6f446c91ae46c: start","job_status":"start","queue_duration_s":1.204708,"scheduling_latency_s":1.204731}2091vm-test-run-nixbot-gitlab> gitlab # [  212.523109] gitaly[1387]: time="2026-07-04T13:33:06.777Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAAPG8RY9SEVANFVHM1KR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.769" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2092vm-test-run-nixbot-gitlab> gitlab # [  212.536403] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60904 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK2093vm-test-run-nixbot-gitlab> gitlab # [  212.539152] nixbot[3145]: {"ts": "2026-07-04T13:33:07.180Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"}2094vm-test-run-nixbot-gitlab> gitlab # [  212.542384] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb20d11b1ed6f446c91ae46c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb20d11b1ed6f446c91ae46c: done: 0.015783 sec","job_status":"done","queue_duration_s":1.204708,"scheduling_latency_s":1.204731,"redis_calls":13,"redis_duration_s":0.0030299999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015783,"completed_at":"2026-07-04T13:33:05.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095vm-test-run-nixbot-gitlab> gitlab # [  212.630843] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4de80a34388b5518d9c0760","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4de80a34388b5518d9c0760: start","job_status":"start","queue_duration_s":1.221877,"scheduling_latency_s":1.221897}2096vm-test-run-nixbot-gitlab> gitlab # [  212.632549] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4de80a34388b5518d9c0760","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4de80a34388b5518d9c0760: done: 0.022128 sec","job_status":"done","queue_duration_s":1.221877,"scheduling_latency_s":1.221897,"redis_calls":13,"redis_duration_s":0.002983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022128,"completed_at":"2026-07-04T13:33:05.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097vm-test-run-nixbot-gitlab> gitlab # [  212.706178] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26a38e9d97d84f9d08d1066f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26a38e9d97d84f9d08d1066f: start","job_status":"start","queue_duration_s":1.2453,"scheduling_latency_s":1.245322}2098vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243'2099vm-test-run-nixbot-gitlab> gitlab # [  212.708776] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26a38e9d97d84f9d08d1066f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26a38e9d97d84f9d08d1066f: done: 0.015207 sec","job_status":"done","queue_duration_s":1.2453,"scheduling_latency_s":1.245322,"redis_calls":13,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015207,"completed_at":"2026-07-04T13:33:05.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243', in 0.04 seconds)2101vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-04T13:33:07.055765Z', 'started_at': '2026-07-04T13:33:07.642678Z', 'finished_at': None}]2102vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 7.38 seconds)2103vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses2104vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2105vm-test-run-nixbot-gitlab> gitlab # [  212.808475] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f88c2e1b4bf1dd28eb39c50f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f88c2e1b4bf1dd28eb39c50f: start","job_status":"start","queue_duration_s":1.261866,"scheduling_latency_s":1.261887}2106vm-test-run-nixbot-gitlab> gitlab # [  212.809918] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f88c2e1b4bf1dd28eb39c50f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f88c2e1b4bf1dd28eb39c50f: done: 0.015535 sec","job_status":"done","queue_duration_s":1.261866,"scheduling_latency_s":1.261887,"redis_calls":13,"redis_duration_s":0.002992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015535,"completed_at":"2026-07-04T13:33:05.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2107vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.08 seconds)2108vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2109vm-test-run-nixbot-gitlab> gitlab # [  212.849462] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19225bd29a12bfffd8d52842","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19225bd29a12bfffd8d52842: start","job_status":"start","queue_duration_s":1.27874,"scheduling_latency_s":1.278761}2110vm-test-run-nixbot-gitlab> gitlab # [  212.869396] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19225bd29a12bfffd8d52842","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19225bd29a12bfffd8d52842: done: 0.015478 sec","job_status":"done","queue_duration_s":1.27874,"scheduling_latency_s":1.278761,"redis_calls":13,"redis_duration_s":0.003069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015478,"completed_at":"2026-07-04T13:33:05.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111vm-test-run-nixbot-gitlab> gitlab # [  212.933481] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62c051c0e903fbac988862cb","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62c051c0e903fbac988862cb: start","job_status":"start","queue_duration_s":1.295548,"scheduling_latency_s":1.29557}2112vm-test-run-nixbot-gitlab> gitlab # [  212.953391] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62c051c0e903fbac988862cb","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62c051c0e903fbac988862cb: done: 0.015193 sec","job_status":"done","queue_duration_s":1.295548,"scheduling_latency_s":1.29557,"redis_calls":13,"redis_duration_s":0.003091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015193,"completed_at":"2026-07-04T13:33:05.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2113vm-test-run-nixbot-gitlab> gitlab # [  213.016837] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397dacf4a6c4df35407b87f8","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397dacf4a6c4df35407b87f8: start","job_status":"start","queue_duration_s":1.311982,"scheduling_latency_s":1.312003}2114vm-test-run-nixbot-gitlab> gitlab # [  213.098346] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"397dacf4a6c4df35407b87f8","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-397dacf4a6c4df35407b87f8: done: 0.015203 sec","job_status":"done","queue_duration_s":1.311982,"scheduling_latency_s":1.312003,"redis_calls":13,"redis_duration_s":0.003142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002172,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015203,"completed_at":"2026-07-04T13:33:05.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115vm-test-run-nixbot-gitlab> gitlab # [  213.099833] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a626940753c7de5d7a4c62a0","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a626940753c7de5d7a4c62a0: start","job_status":"start","queue_duration_s":1.328463,"scheduling_latency_s":1.328485}2116vm-test-run-nixbot-gitlab> gitlab # [  213.117081] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a626940753c7de5d7a4c62a0","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a626940753c7de5d7a4c62a0: done: 0.0149 sec","job_status":"done","queue_duration_s":1.328463,"scheduling_latency_s":1.328485,"redis_calls":13,"redis_duration_s":0.003061,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0149,"completed_at":"2026-07-04T13:33:05.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117vm-test-run-nixbot-gitlab> gitlab # [  213.242594] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc872e55e1ab6adbbc4a3afa","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc872e55e1ab6adbbc4a3afa: start","job_status":"start","queue_duration_s":1.345329,"scheduling_latency_s":1.34535}2118vm-test-run-nixbot-gitlab> gitlab # [  213.244201] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc872e55e1ab6adbbc4a3afa","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc872e55e1ab6adbbc4a3afa: done: 0.015055 sec","job_status":"done","queue_duration_s":1.345329,"scheduling_latency_s":1.34535,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-07-04T13:33:05.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119vm-test-run-nixbot-gitlab> gitlab # [  213.261356] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bb35238e319117793b3b26c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bb35238e319117793b3b26c: start","job_status":"start","queue_duration_s":1.36183,"scheduling_latency_s":1.36185}2120vm-test-run-nixbot-gitlab> gitlab # [  213.281138] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bb35238e319117793b3b26c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bb35238e319117793b3b26c: done: 0.015311 sec","job_status":"done","queue_duration_s":1.36183,"scheduling_latency_s":1.36185,"redis_calls":13,"redis_duration_s":0.003239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002301,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015311,"completed_at":"2026-07-04T13:33:05.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121vm-test-run-nixbot-gitlab> gitlab # [  213.387201] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d6d6c7436f6e5edbd441fff","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d6d6c7436f6e5edbd441fff: start","job_status":"start","queue_duration_s":1.378432,"scheduling_latency_s":1.378454}2122vm-test-run-nixbot-gitlab> gitlab # [  213.389163] gitlab-workhorse[1288]: time="2026-07-04T13:33:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672123vm-test-run-nixbot-gitlab> gitlab # [  213.390101] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 26982124vm-test-run-nixbot-gitlab> gitlab # [  213.390644] gitlab-workhorse[1288]: time="2026-07-04T13:33:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3832125vm-test-run-nixbot-gitlab> gitlab # [  213.391790] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 383 "" "git/2.54.0" 1272126vm-test-run-nixbot-gitlab> gitlab # [  213.392915] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 4472127vm-test-run-nixbot-gitlab> gitlab # [  213.394078] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 592128vm-test-run-nixbot-gitlab> gitlab # [  213.394681] gitaly[1387]: time="2026-07-04T13:33:06.777Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAAPG8RY9SEVANFVHM1KR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.769" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2129vm-test-run-nixbot-gitlab> gitlab # [  213.403866] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d6d6c7436f6e5edbd441fff","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d6d6c7436f6e5edbd441fff: done: 0.015091 sec","job_status":"done","queue_duration_s":1.378432,"scheduling_latency_s":1.378454,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015091,"completed_at":"2026-07-04T13:33:05.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130vm-test-run-nixbot-gitlab> gitlab # [  213.532273] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642648b94e1df81df9a0cec8","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642648b94e1df81df9a0cec8: start","job_status":"start","queue_duration_s":1.394828,"scheduling_latency_s":1.394848}2131vm-test-run-nixbot-gitlab> gitlab # [  213.533412] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642648b94e1df81df9a0cec8","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642648b94e1df81df9a0cec8: done: 0.015165 sec","job_status":"done","queue_duration_s":1.394828,"scheduling_latency_s":1.394848,"redis_calls":13,"redis_duration_s":0.003064,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015165,"completed_at":"2026-07-04T13:33:05.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2132vm-test-run-nixbot-gitlab> gitlab # [  213.548344] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebc824ce9470e51e762e722","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebc824ce9470e51e762e722: start","job_status":"start","queue_duration_s":1.411244,"scheduling_latency_s":1.411265}2133vm-test-run-nixbot-gitlab> gitlab # [  213.568399] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebc824ce9470e51e762e722","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebc824ce9470e51e762e722: done: 0.014954 sec","job_status":"done","queue_duration_s":1.411244,"scheduling_latency_s":1.411265,"redis_calls":13,"redis_duration_s":0.002922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014954,"completed_at":"2026-07-04T13:33:05.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134vm-test-run-nixbot-gitlab> gitlab # [  213.677948] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b298c97ca884c340d30f61","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b298c97ca884c340d30f61: start","job_status":"start","queue_duration_s":1.427577,"scheduling_latency_s":1.427598}2135vm-test-run-nixbot-gitlab> gitlab # [  213.679234] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5b298c97ca884c340d30f61","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5b298c97ca884c340d30f61: done: 0.014954 sec","job_status":"done","queue_duration_s":1.427577,"scheduling_latency_s":1.427598,"redis_calls":13,"redis_duration_s":0.003055,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014954,"completed_at":"2026-07-04T13:33:05.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136vm-test-run-nixbot-gitlab> gitlab # [  213.713781] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f35ed2c0c659bd6971cd5da","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f35ed2c0c659bd6971cd5da: start","job_status":"start","queue_duration_s":1.443853,"scheduling_latency_s":1.443873}2137vm-test-run-nixbot-gitlab> gitlab # [  213.733799] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f35ed2c0c659bd6971cd5da","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f35ed2c0c659bd6971cd5da: done: 0.015147 sec","job_status":"done","queue_duration_s":1.443853,"scheduling_latency_s":1.443873,"redis_calls":13,"redis_duration_s":0.003099,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015147,"completed_at":"2026-07-04T13:33:05.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138vm-test-run-nixbot-gitlab> gitlab # [  213.822801] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92266d71a3759fa02f093992","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92266d71a3759fa02f093992: start","job_status":"start","queue_duration_s":1.460273,"scheduling_latency_s":1.460295}2139vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2140vm-test-run-nixbot-gitlab> gitlab # [  213.824203] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92266d71a3759fa02f093992","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92266d71a3759fa02f093992: done: 0.015171 sec","job_status":"done","queue_duration_s":1.460273,"scheduling_latency_s":1.460295,"redis_calls":13,"redis_duration_s":0.003201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015171,"completed_at":"2026-07-04T13:33:05.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141vm-test-run-nixbot-gitlab> gitlab # [  213.877192] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92b6d215ea31f9fc4ad28a84","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92b6d215ea31f9fc4ad28a84: start","job_status":"start","queue_duration_s":1.476847,"scheduling_latency_s":1.476867}2142vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.08 seconds)2143vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2144vm-test-run-nixbot-gitlab> gitlab # [  213.897935] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92b6d215ea31f9fc4ad28a84","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92b6d215ea31f9fc4ad28a84: done: 0.015163 sec","job_status":"done","queue_duration_s":1.476847,"scheduling_latency_s":1.476867,"redis_calls":13,"redis_duration_s":0.003024,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015163,"completed_at":"2026-07-04T13:33:05.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145vm-test-run-nixbot-gitlab> gitlab # [  213.967252] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62427dbda983a5f18fcb04b","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62427dbda983a5f18fcb04b: start","job_status":"start","queue_duration_s":1.493281,"scheduling_latency_s":1.493304}2146vm-test-run-nixbot-gitlab> gitlab # [  213.977924] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62427dbda983a5f18fcb04b","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62427dbda983a5f18fcb04b: done: 0.015102 sec","job_status":"done","queue_duration_s":1.493281,"scheduling_latency_s":1.493304,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015102,"completed_at":"2026-07-04T13:33:05.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147vm-test-run-nixbot-gitlab> gitlab # [  214.041378] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4014b1d3f48a679324f47cf5","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4014b1d3f48a679324f47cf5: start","job_status":"start","queue_duration_s":1.509757,"scheduling_latency_s":1.509777}2148vm-test-run-nixbot-gitlab> gitlab # [  214.061264] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4014b1d3f48a679324f47cf5","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4014b1d3f48a679324f47cf5: done: 0.014764 sec","job_status":"done","queue_duration_s":1.509757,"scheduling_latency_s":1.509777,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014764,"completed_at":"2026-07-04T13:33:05.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149vm-test-run-nixbot-gitlab> gitlab # [  214.124464] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49a30b1fdfe7c19b18f70a51","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49a30b1fdfe7c19b18f70a51: start","job_status":"start","queue_duration_s":1.525883,"scheduling_latency_s":1.525903}2150vm-test-run-nixbot-gitlab> gitlab # [  214.144439] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49a30b1fdfe7c19b18f70a51","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49a30b1fdfe7c19b18f70a51: done: 0.015056 sec","job_status":"done","queue_duration_s":1.525883,"scheduling_latency_s":1.525903,"redis_calls":13,"redis_duration_s":0.0031009999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015056,"completed_at":"2026-07-04T13:33:05.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151vm-test-run-nixbot-gitlab> gitlab # [  214.257211] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bd38784dbfccd600d56fe75","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bd38784dbfccd600d56fe75: start","job_status":"start","queue_duration_s":1.543083,"scheduling_latency_s":1.543105}2152vm-test-run-nixbot-gitlab> gitlab # [  214.259406] nixbot[3145]: {"ts": "2026-07-04T13:33:07.641Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 \"HTTP/1.1 201 Created\""}2153vm-test-run-nixbot-gitlab> gitlab # [  214.260379] nixbot[3145]: INFO:     ::ffff:127.0.0.1:60908 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 200 OK2154vm-test-run-nixbot-gitlab> gitlab # [  214.261610] gitaly[1387]: time="2026-07-04T13:33:06.777Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAAPG8RY9SEVANFVHM1KR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.769" grpc.time_ms=8.266 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2155vm-test-run-nixbot-gitlab> gitlab # [  214.262825] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bd38784dbfccd600d56fe75","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bd38784dbfccd600d56fe75: done: 0.015781 sec","job_status":"done","queue_duration_s":1.543083,"scheduling_latency_s":1.543105,"redis_calls":13,"redis_duration_s":0.0033290000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015781,"completed_at":"2026-07-04T13:33:05.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156vm-test-run-nixbot-gitlab> gitlab # [  214.320079] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"895368d8749f97c99ab52da6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-895368d8749f97c99ab52da6: start","job_status":"start","queue_duration_s":1.560182,"scheduling_latency_s":1.560202}2157vm-test-run-nixbot-gitlab> gitlab # [  214.401303] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"895368d8749f97c99ab52da6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-895368d8749f97c99ab52da6: done: 0.01546 sec","job_status":"done","queue_duration_s":1.560182,"scheduling_latency_s":1.560202,"redis_calls":13,"redis_duration_s":0.0032170000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002294,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01546,"completed_at":"2026-07-04T13:33:05.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158vm-test-run-nixbot-gitlab> gitlab # [  214.403162] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75787b7c1fd5c8f04588b055","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75787b7c1fd5c8f04588b055: start","job_status":"start","queue_duration_s":1.576903,"scheduling_latency_s":1.576924}2159vm-test-run-nixbot-gitlab> gitlab # [  214.420402] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75787b7c1fd5c8f04588b055","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75787b7c1fd5c8f04588b055: done: 0.014963 sec","job_status":"done","queue_duration_s":1.576903,"scheduling_latency_s":1.576924,"redis_calls":13,"redis_duration_s":0.002957,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014963,"completed_at":"2026-07-04T13:33:05.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160vm-test-run-nixbot-gitlab> gitlab # [  214.483859] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7e982a512ce646f12c4c6da","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7e982a512ce646f12c4c6da: start","job_status":"start","queue_duration_s":1.5932,"scheduling_latency_s":1.593221}2161vm-test-run-nixbot-gitlab> gitlab # [  214.503757] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7e982a512ce646f12c4c6da","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7e982a512ce646f12c4c6da: done: 0.014919 sec","job_status":"done","queue_duration_s":1.5932,"scheduling_latency_s":1.593221,"redis_calls":13,"redis_duration_s":0.002903,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-07-04T13:33:05.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162vm-test-run-nixbot-gitlab> gitlab # [  214.567048] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:05.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24c1b5309b9dba8df0eb6e11","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24c1b5309b9dba8df0eb6e11: start","job_status":"start","queue_duration_s":1.609446,"scheduling_latency_s":1.609467}2163vm-test-run-nixbot-gitlab> gitlab # [  214.586916] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24c1b5309b9dba8df0eb6e11","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24c1b5309b9dba8df0eb6e11: done: 0.015001 sec","job_status":"done","queue_duration_s":1.609446,"scheduling_latency_s":1.609467,"redis_calls":13,"redis_duration_s":0.003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015001,"completed_at":"2026-07-04T13:33:06.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164vm-test-run-nixbot-gitlab> gitlab # [  214.650190] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1443d2d67666d9f276fd82c4","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1443d2d67666d9f276fd82c4: start","job_status":"start","queue_duration_s":1.625837,"scheduling_latency_s":1.625859}2165vm-test-run-nixbot-gitlab> gitlab # [  214.670073] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1443d2d67666d9f276fd82c4","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1443d2d67666d9f276fd82c4: done: 0.01498 sec","job_status":"done","queue_duration_s":1.625837,"scheduling_latency_s":1.625859,"redis_calls":13,"redis_duration_s":0.003056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-07-04T13:33:06.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166vm-test-run-nixbot-gitlab> gitlab # [  214.733329] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eff190c373e4e52b2f3de2f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eff190c373e4e52b2f3de2f: start","job_status":"start","queue_duration_s":1.642087,"scheduling_latency_s":1.642107}2167vm-test-run-nixbot-gitlab> gitlab # [  214.753216] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eff190c373e4e52b2f3de2f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eff190c373e4e52b2f3de2f: done: 0.015097 sec","job_status":"done","queue_duration_s":1.642087,"scheduling_latency_s":1.642107,"redis_calls":13,"redis_duration_s":0.003161,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015097,"completed_at":"2026-07-04T13:33:06.038Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168vm-test-run-nixbot-gitlab> gitlab # [  214.840484] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20a1238a7dbee50c8425d93f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20a1238a7dbee50c8425d93f: start","job_status":"start","queue_duration_s":1.658464,"scheduling_latency_s":1.658484}2169vm-test-run-nixbot-gitlab> gitlab # [  214.841820] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20a1238a7dbee50c8425d93f","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20a1238a7dbee50c8425d93f: done: 0.015051 sec","job_status":"done","queue_duration_s":1.658464,"scheduling_latency_s":1.658484,"redis_calls":13,"redis_duration_s":0.003261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015051,"completed_at":"2026-07-04T13:33:06.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170vm-test-run-nixbot-gitlab> gitlab # [  214.896875] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea531d37b3baa03b88f0269e","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea531d37b3baa03b88f0269e: start","job_status":"start","queue_duration_s":1.67483,"scheduling_latency_s":1.67485}2171vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2172vm-test-run-nixbot-gitlab> gitlab # [  214.916507] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea531d37b3baa03b88f0269e","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea531d37b3baa03b88f0269e: done: 0.015114 sec","job_status":"done","queue_duration_s":1.67483,"scheduling_latency_s":1.67485,"redis_calls":13,"redis_duration_s":0.003104,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-07-04T13:33:06.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2173vm-test-run-nixbot-gitlab> gitlab # [  214.979971] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c60d5684dcf8efbe568c7d5b","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c60d5684dcf8efbe568c7d5b: start","job_status":"start","queue_duration_s":1.691255,"scheduling_latency_s":1.691275}2174vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.12 seconds)2175vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2176vm-test-run-nixbot-gitlab> gitlab # [  214.999916] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c60d5684dcf8efbe568c7d5b","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c60d5684dcf8efbe568c7d5b: done: 0.014994 sec","job_status":"done","queue_duration_s":1.691255,"scheduling_latency_s":1.691275,"redis_calls":13,"redis_duration_s":0.0029890000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014994,"completed_at":"2026-07-04T13:33:06.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177vm-test-run-nixbot-gitlab> gitlab # [  215.091253] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"247acf97ec7afed8f292bc71","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-247acf97ec7afed8f292bc71: start","job_status":"start","queue_duration_s":1.70759,"scheduling_latency_s":1.707611}2178vm-test-run-nixbot-gitlab> gitlab # [  215.092928] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 502179vm-test-run-nixbot-gitlab> gitlab # [  215.094176] gitaly[1387]: time="2026-07-04T13:33:06.908Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADAYGYRDPKTFD41K5Z7V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.895" pid=1387 remote_ip="2001:db8:1::1" request_sha=d196e0c99f60bdd688cfec2b4e953e09dbf8ac24 response_bytes=383 span.kind=server system=grpc2180vm-test-run-nixbot-gitlab> gitlab # [  215.101128] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"247acf97ec7afed8f292bc71","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-247acf97ec7afed8f292bc71: done: 0.014947 sec","job_status":"done","queue_duration_s":1.70759,"scheduling_latency_s":1.707611,"redis_calls":13,"redis_duration_s":0.003038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014947,"completed_at":"2026-07-04T13:33:06.104Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181vm-test-run-nixbot-gitlab> gitlab # [  215.164557] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93da263d79935104a5f603ef","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93da263d79935104a5f603ef: start","job_status":"start","queue_duration_s":1.723845,"scheduling_latency_s":1.723866}2182vm-test-run-nixbot-gitlab> gitlab # [  215.233816] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93da263d79935104a5f603ef","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93da263d79935104a5f603ef: done: 0.015139 sec","job_status":"done","queue_duration_s":1.723845,"scheduling_latency_s":1.723866,"redis_calls":13,"redis_duration_s":0.0031149999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015139,"completed_at":"2026-07-04T13:33:06.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183vm-test-run-nixbot-gitlab> gitlab # [  215.245161] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139a74dbb52cae85df46ebfe","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139a74dbb52cae85df46ebfe: start","job_status":"start","queue_duration_s":1.740276,"scheduling_latency_s":1.740296}2184vm-test-run-nixbot-gitlab> gitlab # [  215.265134] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"139a74dbb52cae85df46ebfe","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-139a74dbb52cae85df46ebfe: done: 0.015028 sec","job_status":"done","queue_duration_s":1.740276,"scheduling_latency_s":1.740296,"redis_calls":13,"redis_duration_s":0.0030960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015028,"completed_at":"2026-07-04T13:33:06.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185vm-test-run-nixbot-gitlab> gitlab # [  215.328948] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49d9917b2189b2d701bb6dc1","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49d9917b2189b2d701bb6dc1: start","job_status":"start","queue_duration_s":1.756684,"scheduling_latency_s":1.756705}2186vm-test-run-nixbot-gitlab> gitlab # [  215.348813] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49d9917b2189b2d701bb6dc1","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49d9917b2189b2d701bb6dc1: done: 0.015098 sec","job_status":"done","queue_duration_s":1.756684,"scheduling_latency_s":1.756705,"redis_calls":13,"redis_duration_s":0.003156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015098,"completed_at":"2026-07-04T13:33:06.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187vm-test-run-nixbot-gitlab> gitlab # [  215.421997] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f32d1bbbefd6c4f6306294f9","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f32d1bbbefd6c4f6306294f9: start","job_status":"start","queue_duration_s":1.773061,"scheduling_latency_s":1.773081}2188vm-test-run-nixbot-gitlab> gitlab # [  215.428996] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f32d1bbbefd6c4f6306294f9","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f32d1bbbefd6c4f6306294f9: done: 0.014957 sec","job_status":"done","queue_duration_s":1.773061,"scheduling_latency_s":1.773081,"redis_calls":13,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-07-04T13:33:06.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189vm-test-run-nixbot-gitlab> gitlab # [  215.492890] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5c5d5324aae7ce9e30ee088","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5c5d5324aae7ce9e30ee088: start","job_status":"start","queue_duration_s":1.789315,"scheduling_latency_s":1.789336}2190vm-test-run-nixbot-gitlab> gitlab # [  215.562985] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5c5d5324aae7ce9e30ee088","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5c5d5324aae7ce9e30ee088: done: 0.015066 sec","job_status":"done","queue_duration_s":1.789315,"scheduling_latency_s":1.789336,"redis_calls":13,"redis_duration_s":0.003054,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015066,"completed_at":"2026-07-04T13:33:06.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191vm-test-run-nixbot-gitlab> gitlab # [  215.574042] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4443b9fcea5d45b25c2a914","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4443b9fcea5d45b25c2a914: start","job_status":"start","queue_duration_s":1.805759,"scheduling_latency_s":1.805779}2192vm-test-run-nixbot-gitlab> gitlab # [  215.593995] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4443b9fcea5d45b25c2a914","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4443b9fcea5d45b25c2a914: done: 0.015024 sec","job_status":"done","queue_duration_s":1.805759,"scheduling_latency_s":1.805779,"redis_calls":13,"redis_duration_s":0.0031390000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002246,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015024,"completed_at":"2026-07-04T13:33:06.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193vm-test-run-nixbot-gitlab> gitlab # [  215.708147] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77e8b7f4a1d853083f0cf2f6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77e8b7f4a1d853083f0cf2f6: start","job_status":"start","queue_duration_s":1.822123,"scheduling_latency_s":1.822143}2194vm-test-run-nixbot-gitlab> gitlab # [  215.709385] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77e8b7f4a1d853083f0cf2f6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77e8b7f4a1d853083f0cf2f6: done: 0.015101 sec","job_status":"done","queue_duration_s":1.822123,"scheduling_latency_s":1.822143,"redis_calls":13,"redis_duration_s":0.0031079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015101,"completed_at":"2026-07-04T13:33:06.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195vm-test-run-nixbot-gitlab> gitlab # [  215.777285] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75a34abae043627349e74a07","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75a34abae043627349e74a07: start","job_status":"start","queue_duration_s":1.838581,"scheduling_latency_s":1.838602}2196vm-test-run-nixbot-gitlab> gitlab # [  215.778346] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75a34abae043627349e74a07","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75a34abae043627349e74a07: done: 0.015112 sec","job_status":"done","queue_duration_s":1.838581,"scheduling_latency_s":1.838602,"redis_calls":13,"redis_duration_s":0.0029249999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015112,"completed_at":"2026-07-04T13:33:06.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197vm-test-run-nixbot-gitlab> gitlab # [  215.851414] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb328adfb34a967b3eb7753c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb328adfb34a967b3eb7753c: start","job_status":"start","queue_duration_s":1.854973,"scheduling_latency_s":1.854993}2198vm-test-run-nixbot-gitlab> gitlab # [  215.852869] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb328adfb34a967b3eb7753c","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb328adfb34a967b3eb7753c: done: 0.015774 sec","job_status":"done","queue_duration_s":1.854973,"scheduling_latency_s":1.854993,"redis_calls":13,"redis_duration_s":0.0031940000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015774,"completed_at":"2026-07-04T13:33:06.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199vm-test-run-nixbot-gitlab> gitlab # [  215.900090] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c75b8d2c86269b0025591f6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c75b8d2c86269b0025591f6: start","job_status":"start","queue_duration_s":1.872059,"scheduling_latency_s":1.872081}2200vm-test-run-nixbot-gitlab> gitlab # [  215.920522] gitaly[1387]: time="2026-07-04T13:33:06.909Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADAYGYRDPKTFD41K5Z7V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.895" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2201vm-test-run-nixbot-gitlab> gitlab # [  215.994966] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 632202vm-test-run-nixbot-gitlab> gitlab # [  215.996316] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c75b8d2c86269b0025591f6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c75b8d2c86269b0025591f6: done: 0.015271 sec","job_status":"done","queue_duration_s":1.872059,"scheduling_latency_s":1.872081,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015271,"completed_at":"2026-07-04T13:33:06.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2203vm-test-run-nixbot-gitlab> gitlab # [  216.007000] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d11ac4911312f79ab3762a9","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d11ac4911312f79ab3762a9: start","job_status":"start","queue_duration_s":1.888762,"scheduling_latency_s":1.888782}2204vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2205vm-test-run-nixbot-gitlab> gitlab # [  216.026997] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d11ac4911312f79ab3762a9","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d11ac4911312f79ab3762a9: done: 0.015352 sec","job_status":"done","queue_duration_s":1.888762,"scheduling_latency_s":1.888782,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015352,"completed_at":"2026-07-04T13:33:06.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206vm-test-run-nixbot-gitlab> gitlab # [  216.138211] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc1ac77437efa5f845b76a9a","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc1ac77437efa5f845b76a9a: start","job_status":"start","queue_duration_s":1.905453,"scheduling_latency_s":1.905474}2207vm-test-run-nixbot-gitlab> gitlab # [  216.139798] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc1ac77437efa5f845b76a9a","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc1ac77437efa5f845b76a9a: done: 0.015037 sec","job_status":"done","queue_duration_s":1.905453,"scheduling_latency_s":1.905474,"redis_calls":13,"redis_duration_s":0.002974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015037,"completed_at":"2026-07-04T13:33:06.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208vm-test-run-nixbot-gitlab> gitlab # [  216.209043] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d89c56db32915cbda823b6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d89c56db32915cbda823b6: start","job_status":"start","queue_duration_s":1.921846,"scheduling_latency_s":1.921866}2209vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.18 seconds)2210vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2211vm-test-run-nixbot-gitlab> gitlab # [  216.210086] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77d89c56db32915cbda823b6","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77d89c56db32915cbda823b6: done: 0.014981 sec","job_status":"done","queue_duration_s":1.921846,"scheduling_latency_s":1.921866,"redis_calls":13,"redis_duration_s":0.003053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014981,"completed_at":"2026-07-04T13:33:06.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212vm-test-run-nixbot-gitlab> gitlab # [  216.280407] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3078d74a93eda64a8b5761fa","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3078d74a93eda64a8b5761fa: start","job_status":"start","queue_duration_s":1.938212,"scheduling_latency_s":1.938233}2213vm-test-run-nixbot-gitlab> gitlab # [  216.281952] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3078d74a93eda64a8b5761fa","created_at":"2026-07-04T13:33:04.317Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-2b09c1b82c414cba","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b803337ac50e23a5e258cec48d8cce8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.381Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3078d74a93eda64a8b5761fa: done: 0.015018 sec","job_status":"done","queue_duration_s":1.938212,"scheduling_latency_s":1.938233,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015018,"completed_at":"2026-07-04T13:33:06.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214vm-test-run-nixbot-gitlab> gitlab # [  216.351999] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.336Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e4017fa9b84ef9ed4af40b0d","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e4017fa9b84ef9ed4af40b0d: start","job_status":"start","queue_duration_s":1.605892,"scheduling_latency_s":1.605913}2215vm-test-run-nixbot-gitlab> gitlab # [  216.353102] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.346Z","message":"No existing merge request to be cleaned up."}2216vm-test-run-nixbot-gitlab> gitlab # [  216.353886] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.353Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e4017fa9b84ef9ed4af40b0d","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e4017fa9b84ef9ed4af40b0d: done: 0.017156 sec","job_status":"done","queue_duration_s":1.605892,"scheduling_latency_s":1.605913,"redis_calls":10,"redis_duration_s":0.002509,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001578,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017156,"completed_at":"2026-07-04T13:33:06.353Z","load_balancing_strategy":"primary","db_duration_s":0.00212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217vm-test-run-nixbot-gitlab> gitlab # [  216.422209] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.354Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"113fe9ef0cb8d5effe9bc8d0","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-113fe9ef0cb8d5effe9bc8d0: start","job_status":"start","queue_duration_s":1.624383,"scheduling_latency_s":1.624404}2218vm-test-run-nixbot-gitlab> gitlab # [  216.430400] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.367Z","message":"No existing merge request to be cleaned up."}2219vm-test-run-nixbot-gitlab> gitlab # [  216.432912] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.374Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"113fe9ef0cb8d5effe9bc8d0","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-113fe9ef0cb8d5effe9bc8d0: done: 0.019183 sec","job_status":"done","queue_duration_s":1.624383,"scheduling_latency_s":1.624404,"redis_calls":10,"redis_duration_s":0.002587,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019183,"completed_at":"2026-07-04T13:33:06.373Z","load_balancing_strategy":"primary","db_duration_s":0.007135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220vm-test-run-nixbot-gitlab> gitlab # [  216.495249] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.375Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5589a5b37c04f18df18db0c","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5589a5b37c04f18df18db0c: start","job_status":"start","queue_duration_s":1.644942,"scheduling_latency_s":1.644964}2221vm-test-run-nixbot-gitlab> gitlab # [  216.565104] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.384Z","message":"No existing merge request to be cleaned up."}2222vm-test-run-nixbot-gitlab> gitlab # [  216.566151] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.390Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e5589a5b37c04f18df18db0c","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e5589a5b37c04f18df18db0c: done: 0.015346 sec","job_status":"done","queue_duration_s":1.644942,"scheduling_latency_s":1.644964,"redis_calls":10,"redis_duration_s":0.002357,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015346,"completed_at":"2026-07-04T13:33:06.390Z","load_balancing_strategy":"primary","db_duration_s":0.001561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223vm-test-run-nixbot-gitlab> gitlab # [  216.577311] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.392Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b06fedb35eaedca58b3f1e7","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b06fedb35eaedca58b3f1e7: start","job_status":"start","queue_duration_s":1.661615,"scheduling_latency_s":1.661635}2224vm-test-run-nixbot-gitlab> gitlab # [  216.662049] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.401Z","message":"No existing merge request to be cleaned up."}2225vm-test-run-nixbot-gitlab> gitlab # [  216.663077] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:06.407Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b06fedb35eaedca58b3f1e7","created_at":"2026-07-04T13:33:04.721Z","trace_propagation_headers":{"sentry-trace":"11a91ec9805543659a3d7cb69f27e9d8-b77ce44338594713","baggage":"sentry-trace_id=11a91ec9805543659a3d7cb69f27e9d8,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"ae88ed461093a1f8d9f9a11f431637ba","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T13:33:04.730Z","job_size_bytes":2,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b06fedb35eaedca58b3f1e7: done: 0.015303 sec","job_status":"done","queue_duration_s":1.661615,"scheduling_latency_s":1.661635,"redis_calls":10,"redis_duration_s":0.0023769999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001496,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015303,"completed_at":"2026-07-04T13:33:06.407Z","load_balancing_strategy":"primary","db_duration_s":0.001455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226vm-test-run-nixbot-gitlab> gitlab # [  216.664275] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.586Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d4324742bec3b0e4661328a7","created_at":"2026-07-04T13:33:07.581Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.584Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d4324742bec3b0e4661328a7: start","job_status":"start","queue_duration_s":0.001702,"scheduling_latency_s":0.001749}2227vm-test-run-nixbot-gitlab> gitlab # [  216.686166] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.612Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d4324742bec3b0e4661328a7","created_at":"2026-07-04T13:33:07.581Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.584Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d4324742bec3b0e4661328a7: done: 0.026523 sec","job_status":"done","queue_duration_s":0.001702,"scheduling_latency_s":0.001749,"redis_calls":7,"redis_duration_s":0.001867,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026523,"completed_at":"2026-07-04T13:33:07.612Z","load_balancing_strategy":"primary","db_duration_s":0.00263,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2228vm-test-run-nixbot-gitlab> gitlab # [  216.793161] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.614Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d00652c3c4da83c8071cdcdd","created_at":"2026-07-04T13:33:07.585Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.586Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d00652c3c4da83c8071cdcdd: start","job_status":"start","queue_duration_s":0.027524,"scheduling_latency_s":0.027591}2229vm-test-run-nixbot-gitlab> gitlab # [  216.794881] gitaly[1387]: time="2026-07-04T13:33:06.909Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADAYGYRDPKTFD41K5Z7V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:06.895" grpc.time_ms=13.931 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2230vm-test-run-nixbot-gitlab> gitlab # [  216.804578] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1362231vm-test-run-nixbot-gitlab> gitlab # [  216.808452] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.663Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d00652c3c4da83c8071cdcdd","created_at":"2026-07-04T13:33:07.585Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.586Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d00652c3c4da83c8071cdcdd: done: 0.049109 sec","job_status":"done","queue_duration_s":0.027524,"scheduling_latency_s":0.027591,"redis_calls":10,"redis_duration_s":0.003118,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049109,"completed_at":"2026-07-04T13:33:07.663Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2232vm-test-run-nixbot-gitlab> gitlab # [  216.879619] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.664Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fcd6b81ed161e2045802e65f","created_at":"2026-07-04T13:33:07.626Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.629Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fcd6b81ed161e2045802e65f: start","job_status":"start","queue_duration_s":0.035776,"scheduling_latency_s":0.035802}2233vm-test-run-nixbot-gitlab> gitlab # [  216.904910] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.785Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fcd6b81ed161e2045802e65f","created_at":"2026-07-04T13:33:07.626Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-1183471dabc84b8a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.629Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fcd6b81ed161e2045802e65f: done: 0.120516 sec","job_status":"done","queue_duration_s":0.035776,"scheduling_latency_s":0.035802,"gitaly_calls":3,"gitaly_duration_s":0.010838,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007601,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00112,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001747,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000375,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001571,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000235,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000437,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072046,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10105167,"duration_s":0.120516,"completed_at":"2026-07-04T13:33:07.785Z","load_balancing_strategy":"primary","db_duration_s":0.030665,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2234vm-test-run-nixbot-gitlab> gitlab # [  216.999212] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f91ef41ec527fa54714deabf","created_at":"2026-07-04T13:33:07.705Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-efb32faf436a467a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.707Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f91ef41ec527fa54714deabf: start","job_status":"start","queue_duration_s":0.07973,"scheduling_latency_s":0.079751}2235vm-test-run-nixbot-gitlab> gitlab # [  217.010716] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:07.863Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f91ef41ec527fa54714deabf","created_at":"2026-07-04T13:33:07.705Z","trace_propagation_headers":{"sentry-trace":"dbcf64bdc35b449d89377a7ce5fe2e6c-efb32faf436a467a","baggage":"sentry-trace_id=dbcf64bdc35b449d89377a7ce5fe2e6c,sentry-release=Unknown"},"correlation_id":"01KWPNADQRN17Q1TM9AWV5MC1R","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:07.707Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f91ef41ec527fa54714deabf: done: 0.076573 sec","job_status":"done","queue_duration_s":0.07973,"scheduling_latency_s":0.079751,"redis_calls":7,"redis_duration_s":0.003829,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003366,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076573,"completed_at":"2026-07-04T13:33:07.863Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236vm-test-run-nixbot-gitlab> gitlab # [  217.142666] gitaly[1387]: time="2026-07-04T13:33:07.319Z" level=info msg="finished unary call with code OK" catfile.duration_ms=45 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=45 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:37.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.273" grpc.time_ms=45.77 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237vm-test-run-nixbot-gitlab> gitlab # [  217.143754] gitaly[1387]: time="2026-07-04T13:33:07.340Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-04T13:33:37.131" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:07.332" grpc.time_ms=8.467 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238vm-test-run-nixbot-gitlab> gitlab # [  217.144867] gitaly[1387]: time="2026-07-04T13:33:07.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:17.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:07.346" grpc.time_ms=6.986 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2239vm-test-run-nixbot-gitlab> gitlab # [  217.146045] gitaly[1387]: time="2026-07-04T13:33:07.473Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T13:33:17.002" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:07.473" grpc.time_ms=0.312 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2240vm-test-run-nixbot-gitlab> gitlab # [  217.159237] gitaly[1387]: time="2026-07-04T13:33:07.485Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T13:33:17.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:07.478" grpc.time_ms=7.736 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2241vm-test-run-nixbot-gitlab> gitlab # [  217.180854] gitaly[1387]: time="2026-07-04T13:33:07.521Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:07.521" grpc.time_ms=0.442 pid=1387 span.kind=server system=grpc2242vm-test-run-nixbot-gitlab> gitlab # [  217.196076] gitaly[1387]: time="2026-07-04T13:33:07.562Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:07.562" grpc.time_ms=0.552 pid=1387 span.kind=server system=grpc2243vm-test-run-nixbot-gitlab> gitlab # [  217.212057] gitaly[1387]: time="2026-07-04T13:33:07.575Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R diskcache=86e7fc1a-3ef4-498e-b7f4-2c8a47ce6664 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:17.017" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:07.487" pid=1387 ref=refs/keep-around/fe2a76c9ff364b509bf59bd688e036c631d88243 remote_ip="2001:db8:1::1" revision=fe2a76c9ff364b509bf59bd688e036c631d88243 span.kind=server system=grpc username=root2244vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2245vm-test-run-nixbot-gitlab> gitlab # [  217.286365] gitaly[1387]: time="2026-07-04T13:33:07.575Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=64 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5198 command.oublock=8 command.real_time_ms=79 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T13:33:17.017" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T13:33:07.487" grpc.time_ms=87.343 pid=1387 ref=refs/keep-around/fe2a76c9ff364b509bf59bd688e036c631d88243 remote_ip="2001:db8:1::1" revision=fe2a76c9ff364b509bf59bd688e036c631d88243 span.kind=server system=grpc username=root2246vm-test-run-nixbot-gitlab> gitlab # [  217.322616] nixbot[3145]: {"ts": "2026-07-04T13:33:12.268Z", "level": "warning", "logger": "nixbot.nix_eval", "msg": "failed to remove eval cgroup", "path": "/sys/fs/cgroup/system.slice/nixbot.service/eval-0"}2247vm-test-run-nixbot-gitlab> gitlab # [  217.323391] nixbot[3145]: {"ts": "2026-07-04T13:33:12.270Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-fe2a76c9ff36-b8bf5d5a", "argv0": "bwrap"}2248vm-test-run-nixbot-gitlab> gitlab # [  217.324290] gitaly[1387]: time="2026-07-04T13:33:07.579Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:37.378" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.579" grpc.time_ms=0.415 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2249vm-test-run-nixbot-gitlab> gitlab # [  217.325193] gitaly[1387]: time="2026-07-04T13:33:07.716Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:37.314" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.715" grpc.time_ms=0.993 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2250vm-test-run-nixbot-gitlab> gitlab # [  217.326070] gitaly[1387]: time="2026-07-04T13:33:07.756Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:37.354" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.755" grpc.time_ms=0.817 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2251vm-test-run-nixbot-gitlab> gitlab # [  217.326893] gitaly[1387]: time="2026-07-04T13:33:07.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNADQRN17Q1TM9AWV5MC1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:37.375" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.776" grpc.time_ms=0.543 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2252vm-test-run-nixbot-gitlab> gitlab # [  217.339819] gitaly[1387]: time="2026-07-04T13:33:07.912Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAED1H644AYJN3FF6PG0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:37.711" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:07.911" grpc.time_ms=0.617 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2253vm-test-run-nixbot-gitlab> gitlab # [  217.360080] gitaly[1387]: time="2026-07-04T13:33:09.003Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAFF7DXT2741HVKKGK87Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:39.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:09.002" grpc.time_ms=0.857 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2254vm-test-run-nixbot-gitlab> gitlab # [  217.380887] gitaly[1387]: time="2026-07-04T13:33:10.115Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAGHNP14GYCZW9YKMB8S6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:40.114" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:10.114" grpc.time_ms=1.005 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2255vm-test-run-nixbot-gitlab> gitlab # [  217.453197] gitaly[1387]: time="2026-07-04T13:33:11.209Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAHM330YZZB1GC8EMB20N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:41.008" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:11.208" grpc.time_ms=0.839 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2256vm-test-run-nixbot-gitlab> gitlab # [  217.505620] gitaly[1387]: time="2026-07-04T13:33:12.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJRSACKMNTFZ100RXR16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:42.516" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:12.516" grpc.time_ms=0.592 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2257vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.25 seconds)2258vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2259vm-test-run-nixbot-gitlab> gitlab # [  217.549252] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 2152260vm-test-run-nixbot-gitlab> gitlab # [  217.567790] gitaly[1387]: time="2026-07-04T13:33:12.579Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:42.378" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:12.578" grpc.time_ms=0.825 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2261vm-test-run-nixbot-gitlab> gitlab # [  217.578254] nix-daemon[3393]: accepted connection from pid 3534, user nixbot2262vm-test-run-nixbot-gitlab> gitlab # [  217.584890] nix-daemon[3393]: accepted connection from pid 3535, user nixbot2263vm-test-run-nixbot-gitlab> gitlab # [  217.646306] gitaly[1387]: time="2026-07-04T13:33:12.658Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:22.118" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:12.648" grpc.time_ms=9.07 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2264vm-test-run-nixbot-gitlab> gitlab # [  217.724671] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:12.735Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ceadea41f5064e2a1ecedaef","created_at":"2026-07-04T13:33:12.728Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-30b7556075f84b80","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:12.733Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ceadea41f5064e2a1ecedaef: start","job_status":"start","queue_duration_s":0.002345,"scheduling_latency_s":0.002403}2265vm-test-run-nixbot-gitlab> gitlab # [  217.768424] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 2252266vm-test-run-nixbot-gitlab> gitlab # [  217.769551] nixbot[3145]: {"ts": "2026-07-04T13:33:12.753Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 \"HTTP/1.1 201 Created\""}2267vm-test-run-nixbot-gitlab> gitlab # [  217.810440] gitaly[1387]: time="2026-07-04T13:33:12.822Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAK5YEH7AQ9K1V9WHGEB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:42.821" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:12.821" grpc.time_ms=0.786 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2268vm-test-run-nixbot-gitlab> gitlab # [  217.859235] gitaly[1387]: time="2026-07-04T13:33:12.871Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAK5YEH7AQ9K1V9WHGEB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:22.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:12.864" grpc.time_ms=6.497 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2269vm-test-run-nixbot-gitlab> gitlab # [  217.886159] gitaly[1387]: time="2026-07-04T13:33:12.898Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:12.897" grpc.time_ms=0.501 pid=1387 span.kind=server system=grpc2270vm-test-run-nixbot-gitlab> gitlab # [  217.926148] gitaly[1387]: time="2026-07-04T13:33:12.938Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:12.936" grpc.time_ms=1.499 pid=1387 span.kind=server system=grpc2271vm-test-run-nixbot-gitlab> gitlab # [  217.949316] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 1932272vm-test-run-nixbot-gitlab> gitlab # [  217.949886] nixbot[3145]: {"ts": "2026-07-04T13:33:12.961Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2167be41b58e6269099e2a2588e308eb35c22060 \"HTTP/1.1 201 Created\""}2273vm-test-run-nixbot-gitlab> gitlab # [  217.966973] gitaly[1387]: time="2026-07-04T13:33:12.979Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:12.978" grpc.time_ms=0.348 pid=1387 span.kind=server system=grpc2274vm-test-run-nixbot-gitlab> gitlab # [  217.993073] gitaly[1387]: time="2026-07-04T13:33:13.005Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A diskcache=7e6e155a-f95c-415d-8d5d-a2b14401c2c3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:42.467" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:12.867" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2275vm-test-run-nixbot-gitlab> gitlab # [  217.993963] gitaly[1387]: time="2026-07-04T13:33:13.005Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=103 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7129 command.oublock=0 command.real_time_ms=129 command.spawn_token_fork_ms=2 command.system_time_ms=69 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:42.467" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:12.867" grpc.time_ms=137.295 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2276vm-test-run-nixbot-gitlab> gitlab # [  218.057196] gitaly[1387]: time="2026-07-04T13:33:13.069Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:43.067" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.068" grpc.time_ms=0.864 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2277vm-test-run-nixbot-gitlab> gitlab # [  218.083302] gitaly[1387]: time="2026-07-04T13:33:13.095Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:43.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.094" grpc.time_ms=0.659 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2278vm-test-run-nixbot-gitlab> gitlab # [  218.121828] gitaly[1387]: time="2026-07-04T13:33:13.133Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:43.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.132" grpc.time_ms=1.004 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2279vm-test-run-nixbot-gitlab> gitlab # [  218.131362] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.142Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f33a3a6249df03573fc6f7d2","created_at":"2026-07-04T13:33:13.139Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"ceadea41f5064e2a1ecedaef","job_size_bytes":3,"pid":2137,"job_status":"deduplicated","message":"PipelineProcessWorker JID-f33a3a6249df03573fc6f7d2: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2280vm-test-run-nixbot-gitlab> gitlab # [  218.133241] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.144Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ceadea41f5064e2a1ecedaef","created_at":"2026-07-04T13:33:12.728Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-30b7556075f84b80","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:12.733Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2137,"job_status":"rescheduled","message":"PipelineProcessWorker JID-ceadea41f5064e2a1ecedaef: rescheduled"}2281vm-test-run-nixbot-gitlab> gitlab # [  218.139595] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.151Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"ceadea41f5064e2a1ecedaef","created_at":"2026-07-04T13:33:12.728Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-30b7556075f84b80","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:12.733Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ceadea41f5064e2a1ecedaef: done: 0.41564 sec","job_status":"done","queue_duration_s":0.002345,"scheduling_latency_s":0.002403,"gitaly_calls":4,"gitaly_duration_s":0.148968,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.031789,"redis_read_bytes":264,"redis_write_bytes":24009,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001902,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002716,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.005853,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.004173,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12388,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.010828,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000284,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00062,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003511,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.170006,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.363676389,"duration_s":0.41564,"completed_at":"2026-07-04T13:33:13.151Z","load_balancing_strategy":"primary","db_duration_s":0.068722,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2282vm-test-run-nixbot-gitlab> gitlab # [  218.206395] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ac3584b81ddda821715bb79f","created_at":"2026-07-04T13:33:12.849Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:12.857Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ac3584b81ddda821715bb79f: start","job_status":"start","queue_duration_s":0.295579,"scheduling_latency_s":0.295601}2283vm-test-run-nixbot-gitlab> gitlab # [  218.333241] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ac3584b81ddda821715bb79f","created_at":"2026-07-04T13:33:12.849Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:12.857Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ac3584b81ddda821715bb79f: done: 0.191477 sec","job_status":"done","queue_duration_s":0.295579,"scheduling_latency_s":0.295601,"redis_calls":15,"redis_duration_s":0.007771999999999999,"redis_read_bytes":13,"redis_write_bytes":3011,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00081,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000694,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1626,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00341,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.00096,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.156845,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000509841,"duration_s":0.191477,"completed_at":"2026-07-04T13:33:13.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021262,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2284vm-test-run-nixbot-gitlab> gitlab # [  218.339258] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.346Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"636a9c6340f74c567227a13e","created_at":"2026-07-04T13:33:13.335Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-c0c600c9077f4837","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:bf0908a5adb160f34678201f076cdfc6c4c43b123041154fbc5f9b19510a4f78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.337Z","job_size_bytes":464,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-636a9c6340f74c567227a13e: start","job_status":"start","queue_duration_s":0.009115,"scheduling_latency_s":0.009136}2285vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2286vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.10 seconds)2287vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2288vm-test-run-nixbot-gitlab> gitlab # [  218.374316] sidekiq-cluster[2137]: {"severity":"WARN","time":"2026-07-04T13:33:13.385Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"636a9c6340f74c567227a13e","created_at":"2026-07-04T13:33:13.335Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-c0c600c9077f4837","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:bf0908a5adb160f34678201f076cdfc6c4c43b123041154fbc5f9b19510a4f78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.337Z","job_size_bytes":464,"pid":2137,"sidekiq_tid":"m4dh","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-636a9c6340f74c567227a13e: fail: 0.038587 sec","job_status":"fail","queue_duration_s":0.009115,"scheduling_latency_s":0.009136,"redis_calls":7,"redis_duration_s":0.0029980000000000002,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002236,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038587,"completed_at":"2026-07-04T13:33:13.385Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.010082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2289vm-test-run-nixbot-gitlab> gitlab # [  218.986158] gitaly[1387]: time="2026-07-04T13:33:13.641Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAKZX97JPE9GKD57VH6FJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:43.439" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.640" grpc.time_ms=0.817 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290vm-test-run-nixbot-gitlab> gitlab # [  218.992631] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 612291vm-test-run-nixbot-gitlab> gitlab # [  218.996917] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.388Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"572edc66d35c37baeab87116","created_at":"2026-07-04T13:33:13.010Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.015Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-572edc66d35c37baeab87116: start","job_status":"start","queue_duration_s":0.373235,"scheduling_latency_s":0.373259}2292vm-test-run-nixbot-gitlab> gitlab # [  219.023795] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.523Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"572edc66d35c37baeab87116","created_at":"2026-07-04T13:33:13.010Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.015Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-572edc66d35c37baeab87116: done: 0.1353 sec","job_status":"done","queue_duration_s":0.373235,"scheduling_latency_s":0.373259,"redis_calls":10,"redis_duration_s":0.004267,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000976,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.1,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.096,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000672711,"duration_s":0.1353,"completed_at":"2026-07-04T13:33:13.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.09964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293vm-test-run-nixbot-gitlab> gitlab # [  219.129138] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.526Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d11b14f806bb4b82d547308c","created_at":"2026-07-04T13:33:13.025Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.032Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d11b14f806bb4b82d547308c: start","job_status":"start","queue_duration_s":0.494127,"scheduling_latency_s":0.494183}2294vm-test-run-nixbot-gitlab> gitlab # [  219.130405] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"d11b14f806bb4b82d547308c","created_at":"2026-07-04T13:33:13.025Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.032Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-d11b14f806bb4b82d547308c: done: 0.031202 sec","job_status":"done","queue_duration_s":0.494127,"scheduling_latency_s":0.494183,"redis_calls":10,"redis_duration_s":0.00268,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356749,"duration_s":0.031202,"completed_at":"2026-07-04T13:33:13.557Z","load_balancing_strategy":"primary","db_duration_s":0.005308,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2295vm-test-run-nixbot-gitlab> gitlab # [  219.189479] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.558Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d47f811fe1abb73f20cd0d79","created_at":"2026-07-04T13:33:13.042Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.046Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d47f811fe1abb73f20cd0d79: start","job_status":"start","queue_duration_s":0.511878,"scheduling_latency_s":0.511898}2296vm-test-run-nixbot-gitlab> gitlab # [  219.273298] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d47f811fe1abb73f20cd0d79","created_at":"2026-07-04T13:33:13.042Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.046Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d47f811fe1abb73f20cd0d79: done: 0.035863 sec","job_status":"done","queue_duration_s":0.511878,"scheduling_latency_s":0.511898,"redis_calls":13,"redis_duration_s":0.005309,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000824,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002656,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000387759,"duration_s":0.035863,"completed_at":"2026-07-04T13:33:13.594Z","load_balancing_strategy":"primary","db_duration_s":0.010212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2297vm-test-run-nixbot-gitlab> gitlab # [  219.283999] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.595Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a53ff7fddbab070ac92619e","created_at":"2026-07-04T13:33:13.047Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.050Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a53ff7fddbab070ac92619e: start","job_status":"start","queue_duration_s":0.545169,"scheduling_latency_s":0.54519}2298vm-test-run-nixbot-gitlab> gitlab # [  219.311422] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.631Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6a53ff7fddbab070ac92619e","created_at":"2026-07-04T13:33:13.047Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.050Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6a53ff7fddbab070ac92619e: done: 0.035899 sec","job_status":"done","queue_duration_s":0.545169,"scheduling_latency_s":0.54519,"redis_calls":10,"redis_duration_s":0.003684,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389435,"duration_s":0.035899,"completed_at":"2026-07-04T13:33:13.631Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2299vm-test-run-nixbot-gitlab> gitlab # [  219.418313] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.633Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"490eab316b46535053f312e1","created_at":"2026-07-04T13:33:13.051Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.055Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-490eab316b46535053f312e1: start","job_status":"start","queue_duration_s":0.578875,"scheduling_latency_s":0.578896}2300vm-test-run-nixbot-gitlab> gitlab # [  219.419849] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.662Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"490eab316b46535053f312e1","created_at":"2026-07-04T13:33:13.051Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.055Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-490eab316b46535053f312e1: done: 0.02859 sec","job_status":"done","queue_duration_s":0.578875,"scheduling_latency_s":0.578896,"redis_calls":11,"redis_duration_s":0.004211,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001915,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344457,"duration_s":0.02859,"completed_at":"2026-07-04T13:33:13.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301vm-test-run-nixbot-gitlab> gitlab # [  219.478286] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.664Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c360e94d7d2b83c5ab2cb9e1","created_at":"2026-07-04T13:33:13.057Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.060Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c360e94d7d2b83c5ab2cb9e1: start","job_status":"start","queue_duration_s":0.604212,"scheduling_latency_s":0.604259}2302vm-test-run-nixbot-gitlab> gitlab # [  219.563284] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.694Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c360e94d7d2b83c5ab2cb9e1","created_at":"2026-07-04T13:33:13.057Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.060Z","job_size_bytes":65,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c360e94d7d2b83c5ab2cb9e1: done: 0.029334 sec","job_status":"done","queue_duration_s":0.604212,"scheduling_latency_s":0.604259,"redis_calls":10,"redis_duration_s":0.0023380000000000002,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386362,"duration_s":0.029334,"completed_at":"2026-07-04T13:33:13.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2303vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2304vm-test-run-nixbot-gitlab> gitlab # [  219.577192] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.695Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9294a76fbad928b53b501bb8","created_at":"2026-07-04T13:33:13.061Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.062Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9294a76fbad928b53b501bb8: start","job_status":"start","queue_duration_s":0.632879,"scheduling_latency_s":0.632901}2305vm-test-run-nixbot-gitlab> gitlab # [  219.603250] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.720Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9294a76fbad928b53b501bb8","created_at":"2026-07-04T13:33:13.061Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.062Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9294a76fbad928b53b501bb8: done: 0.025206 sec","job_status":"done","queue_duration_s":0.632879,"scheduling_latency_s":0.632901,"redis_calls":7,"redis_duration_s":0.0017070000000000002,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025206,"completed_at":"2026-07-04T13:33:13.720Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002751,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2306vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.11 seconds)2307vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2308vm-test-run-nixbot-gitlab> gitlab # [  219.734284] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.722Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0e989fea41b1240ce1c0ea98","created_at":"2026-07-04T13:33:13.095Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.100Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0e989fea41b1240ce1c0ea98: start","job_status":"start","queue_duration_s":0.621986,"scheduling_latency_s":0.62201}2309vm-test-run-nixbot-gitlab> gitlab # [  219.735821] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.751Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0e989fea41b1240ce1c0ea98","created_at":"2026-07-04T13:33:13.095Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.100Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0e989fea41b1240ce1c0ea98: done: 0.028849 sec","job_status":"done","queue_duration_s":0.621986,"scheduling_latency_s":0.62201,"redis_calls":10,"redis_duration_s":0.002376,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395302,"duration_s":0.028849,"completed_at":"2026-07-04T13:33:13.750Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2310vm-test-run-nixbot-gitlab> gitlab # [  219.822422] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.752Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d4bdd74ea6f94b9e02bdc861","created_at":"2026-07-04T13:33:13.100Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.105Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d4bdd74ea6f94b9e02bdc861: start","job_status":"start","queue_duration_s":0.646982,"scheduling_latency_s":0.647004}2311vm-test-run-nixbot-gitlab> gitlab # [  219.823899] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d4bdd74ea6f94b9e02bdc861","created_at":"2026-07-04T13:33:13.100Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.105Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d4bdd74ea6f94b9e02bdc861: done: 0.020087 sec","job_status":"done","queue_duration_s":0.646982,"scheduling_latency_s":0.647004,"redis_calls":10,"redis_duration_s":0.00244,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396419,"duration_s":0.020087,"completed_at":"2026-07-04T13:33:13.772Z","load_balancing_strategy":"primary","db_duration_s":0.001978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2312vm-test-run-nixbot-gitlab> gitlab # [  219.860547] gitaly[1387]: time="2026-07-04T13:33:13.808Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:43.406" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.807" grpc.time_ms=0.884 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2313vm-test-run-nixbot-gitlab> gitlab # [  219.942475] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 582314vm-test-run-nixbot-gitlab> gitlab # [  219.943999] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.773Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d124d5f7bc66112990a65c9b","created_at":"2026-07-04T13:33:13.144Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.145Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d124d5f7bc66112990a65c9b: start","job_status":"start","queue_duration_s":0.628003,"scheduling_latency_s":0.628024}2315vm-test-run-nixbot-gitlab> gitlab # [  219.944818] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:13.848Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d124d5f7bc66112990a65c9b","created_at":"2026-07-04T13:33:13.144Z","trace_propagation_headers":{"sentry-trace":"f69dc4e158104640816c9b1693bd30ed-a2ee32db7a5d4990","baggage":"sentry-trace_id=f69dc4e158104640816c9b1693bd30ed,sentry-release=Unknown"},"correlation_id":"01KWPNAJYEV3ASXYTQ0WSM816A","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:13.145Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d124d5f7bc66112990a65c9b: done: 0.075055 sec","job_status":"done","queue_duration_s":0.628003,"scheduling_latency_s":0.628024,"gitaly_calls":2,"gitaly_duration_s":0.006714,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003546,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000684,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001488,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00023,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001144,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05210802,"duration_s":0.075055,"completed_at":"2026-07-04T13:33:13.848Z","load_balancing_strategy":"primary","db_duration_s":0.014103,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2316vm-test-run-nixbot-gitlab> gitlab # [  220.019911] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:14.385Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9d241ea9ce50044db18d0e43","created_at":"2026-07-04T13:32:52.184Z","trace_propagation_headers":{"sentry-trace":"7b0eac1ceee846218476d54c3b673135-bc400282c6884625","baggage":"sentry-trace_id=7b0eac1ceee846218476d54c3b673135,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9YNNASMP0A4WXQMGDYMK","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9ed97a349c1bceda2326a8dad6d333d1d1ac76a447be34d4d32468d96a073f77","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:14.383Z","retry_count":0,"job_size_bytes":451,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9d241ea9ce50044db18d0e43: start","job_status":"start","queue_duration_s":0.002554,"scheduling_latency_s":0.002603}2317vm-test-run-nixbot-gitlab> gitlab # [  220.021235] sidekiq-cluster[2137]: {"severity":"WARN","time":"2026-07-04T13:33:14.419Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9d241ea9ce50044db18d0e43","created_at":"2026-07-04T13:32:52.184Z","trace_propagation_headers":{"sentry-trace":"7b0eac1ceee846218476d54c3b673135-bc400282c6884625","baggage":"sentry-trace_id=7b0eac1ceee846218476d54c3b673135,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPN9YNNASMP0A4WXQMGDYMK","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:9ed97a349c1bceda2326a8dad6d333d1d1ac76a447be34d4d32468d96a073f77","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:14.383Z","retry_count":0,"job_size_bytes":451,"pid":2137,"sidekiq_tid":"m8mp","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9d241ea9ce50044db18d0e43: fail: 0.033683 sec","job_status":"fail","queue_duration_s":0.002554,"scheduling_latency_s":0.002603,"redis_calls":7,"redis_duration_s":0.002864,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033683,"completed_at":"2026-07-04T13:33:14.419Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2318vm-test-run-nixbot-gitlab> gitlab # [  220.610901] gitaly[1387]: time="2026-07-04T13:33:13.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAJYEV3ASXYTQ0WSM816A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:43.436" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:13.836" grpc.time_ms=0.636 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2319vm-test-run-nixbot-gitlab> gitlab # [  220.630052] gitaly[1387]: time="2026-07-04T13:33:14.750Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAN2PSBM79AMFR1W0XFHC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:44.748" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:14.748" grpc.time_ms=1.038 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2320vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2321vm-test-run-nixbot-gitlab> gitlab # [  220.880532] gitaly[1387]: time="2026-07-04T13:33:15.892Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAP5D4WGK9RDPFGK0XBBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:45.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:15.890" grpc.time_ms=1.197 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2322vm-test-run-nixbot-gitlab> gitlab # [  220.901415] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 912323vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.18 seconds)2324vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2325vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2326vm-test-run-nixbot-gitlab> gitlab # [  222.052453] gitaly[1387]: time="2026-07-04T13:33:17.063Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQ9ZJYX60TCRSRBEZWF5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:47.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.062" grpc.time_ms=1.147 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2327vm-test-run-nixbot-gitlab> gitlab # [  222.079297] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 982328vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.17 seconds)2329vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2330vm-test-run-nixbot-gitlab> gitlab # [  222.193479] nix-daemon[3393]: accepted connection from pid 3601, user nixbot2331vm-test-run-nixbot-gitlab> gitlab # [  222.246689] gitaly[1387]: time="2026-07-04T13:33:17.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:47.255" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.256" grpc.time_ms=1.367 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2332vm-test-run-nixbot-gitlab> gitlab # [  222.362300] gitaly[1387]: time="2026-07-04T13:33:17.374Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:27.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:17.359" grpc.time_ms=14.454 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2333vm-test-run-nixbot-gitlab> gitlab # [  222.422652] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.433Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"dc85fc15df1ad1af36dd1e2f","created_at":"2026-07-04T13:33:17.424Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-1662e092168b4eec","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.430Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dc85fc15df1ad1af36dd1e2f: start","job_status":"start","queue_duration_s":0.002769,"scheduling_latency_s":0.002826}2334vm-test-run-nixbot-gitlab> gitlab # [  222.474958] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3092335vm-test-run-nixbot-gitlab> gitlab # [  222.481331] nixbot[3145]: {"ts": "2026-07-04T13:33:17.492Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 \"HTTP/1.1 201 Created\""}2336vm-test-run-nixbot-gitlab> gitlab # [  222.538160] gitaly[1387]: time="2026-07-04T13:33:17.549Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQSSX0KAY7J3K3DVSQ5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:47.546" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.547" grpc.time_ms=1.873 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2337vm-test-run-nixbot-gitlab> gitlab # [  222.549773] gitaly[1387]: time="2026-07-04T13:33:17.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:17.560" grpc.time_ms=0.795 pid=1387 span.kind=server system=grpc2338vm-test-run-nixbot-gitlab> gitlab # [  222.571826] gitaly[1387]: time="2026-07-04T13:33:17.583Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQSSX0KAY7J3K3DVSQ5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:27.104" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:17.574" grpc.time_ms=9.014 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2339vm-test-run-nixbot-gitlab> gitlab # [  222.622221] gitaly[1387]: time="2026-07-04T13:33:17.633Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:17.632" grpc.time_ms=0.52 pid=1387 span.kind=server system=grpc2340vm-test-run-nixbot-gitlab> gitlab # [  222.625726] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1392341vm-test-run-nixbot-gitlab> gitlab # [  222.627076] nixbot[3145]: {"ts": "2026-07-04T13:33:17.638Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 \"HTTP/1.1 201 Created\""}2342vm-test-run-nixbot-gitlab> gitlab # [  222.671119] gitaly[1387]: time="2026-07-04T13:33:17.683Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:17.682" grpc.time_ms=0.458 pid=1387 span.kind=server system=grpc2343vm-test-run-nixbot-gitlab> gitlab # [  222.695945] gitaly[1387]: time="2026-07-04T13:33:17.707Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K diskcache=d4504640-f9ba-4708-807d-ff72fc41f070 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:47.127" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:17.527" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2344vm-test-run-nixbot-gitlab> gitlab # [  222.697100] gitaly[1387]: time="2026-07-04T13:33:17.708Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=107 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7421 command.oublock=0 command.real_time_ms=172 command.spawn_token_fork_ms=0 command.system_time_ms=72 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:47.127" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:17.527" grpc.time_ms=180.235 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2345vm-test-run-nixbot-gitlab> gitlab # [  222.741394] gitaly[1387]: time="2026-07-04T13:33:17.753Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:47.352" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.752" grpc.time_ms=0.887 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2346vm-test-run-nixbot-gitlab> gitlab # [  222.779966] gitaly[1387]: time="2026-07-04T13:33:17.791Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:47.390" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.790" grpc.time_ms=0.736 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2347vm-test-run-nixbot-gitlab> gitlab # [  222.826871] gitaly[1387]: time="2026-07-04T13:33:17.838Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:47.436" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:17.837" grpc.time_ms=0.946 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2348vm-test-run-nixbot-gitlab> gitlab # [  222.836970] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.848Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8108b37f028bd515f319a008","created_at":"2026-07-04T13:33:17.845Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"dc85fc15df1ad1af36dd1e2f","job_size_bytes":3,"pid":2137,"job_status":"deduplicated","message":"PipelineProcessWorker JID-8108b37f028bd515f319a008: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2349vm-test-run-nixbot-gitlab> gitlab # [  222.838874] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.850Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"dc85fc15df1ad1af36dd1e2f","created_at":"2026-07-04T13:33:17.424Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-1662e092168b4eec","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.430Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2137,"job_status":"rescheduled","message":"PipelineProcessWorker JID-dc85fc15df1ad1af36dd1e2f: rescheduled"}2350vm-test-run-nixbot-gitlab> gitlab # [  222.846337] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.857Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"dc85fc15df1ad1af36dd1e2f","created_at":"2026-07-04T13:33:17.424Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-1662e092168b4eec","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.430Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-dc85fc15df1ad1af36dd1e2f: done: 0.424718 sec","job_status":"done","queue_duration_s":0.002769,"scheduling_latency_s":0.002826,"gitaly_calls":4,"gitaly_duration_s":0.192871,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.027829,"redis_read_bytes":262,"redis_write_bytes":23385,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002082,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003569,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001446,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.005048,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.009334,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0005,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000646,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003122,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.012,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.149735,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.37923954,"duration_s":0.424718,"completed_at":"2026-07-04T13:33:17.857Z","load_balancing_strategy":"primary","db_duration_s":0.058405,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2351vm-test-run-nixbot-gitlab> gitlab # [  222.942831] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1ca5c81d1c2a742a485bf3ca","created_at":"2026-07-04T13:33:17.519Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.522Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1ca5c81d1c2a742a485bf3ca: start","job_status":"start","queue_duration_s":0.338154,"scheduling_latency_s":0.338212}2352vm-test-run-nixbot-gitlab> gitlab # [  222.944488] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.903Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1ca5c81d1c2a742a485bf3ca","created_at":"2026-07-04T13:33:17.519Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.522Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1ca5c81d1c2a742a485bf3ca: done: 0.042843 sec","job_status":"done","queue_duration_s":0.338154,"scheduling_latency_s":0.338212,"redis_calls":7,"redis_duration_s":0.002875,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042843,"completed_at":"2026-07-04T13:33:17.903Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011658,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2353vm-test-run-nixbot-gitlab> gitlab # [  223.043314] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.905Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6a8e1980059f6b12b9c5e1a9","created_at":"2026-07-04T13:33:17.713Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.715Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6a8e1980059f6b12b9c5e1a9: start","job_status":"start","queue_duration_s":0.189477,"scheduling_latency_s":0.189505}2354vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2355vm-test-run-nixbot-gitlab> gitlab # [  223.045198] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.930Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6a8e1980059f6b12b9c5e1a9","created_at":"2026-07-04T13:33:17.713Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.715Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6a8e1980059f6b12b9c5e1a9: done: 0.025522 sec","job_status":"done","queue_duration_s":0.189477,"scheduling_latency_s":0.189505,"redis_calls":7,"redis_duration_s":0.0037099999999999998,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002408,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025522,"completed_at":"2026-07-04T13:33:17.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2356vm-test-run-nixbot-gitlab> gitlab # [  223.155132] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a31a073d77923f186558742d","created_at":"2026-07-04T13:33:17.723Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.726Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a31a073d77923f186558742d: start","job_status":"start","queue_duration_s":0.205809,"scheduling_latency_s":0.205831}2357vm-test-run-nixbot-gitlab> gitlab # [  223.156701] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.965Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"a31a073d77923f186558742d","created_at":"2026-07-04T13:33:17.723Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.726Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-a31a073d77923f186558742d: done: 0.032382 sec","job_status":"done","queue_duration_s":0.205809,"scheduling_latency_s":0.205831,"redis_calls":7,"redis_duration_s":0.00316,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002298,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032382,"completed_at":"2026-07-04T13:33:17.965Z","load_balancing_strategy":"primary","db_duration_s":0.009301,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2358vm-test-run-nixbot-gitlab> gitlab # [  223.246120] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.967Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"04f7bbd19a702a59d4bdf55e","created_at":"2026-07-04T13:33:17.735Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.736Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-04f7bbd19a702a59d4bdf55e: start","job_status":"start","queue_duration_s":0.230378,"scheduling_latency_s":0.230403}2359vm-test-run-nixbot-gitlab> gitlab # [  223.247110] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"04f7bbd19a702a59d4bdf55e","created_at":"2026-07-04T13:33:17.735Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.736Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-04f7bbd19a702a59d4bdf55e: done: 0.0297 sec","job_status":"done","queue_duration_s":0.230378,"scheduling_latency_s":0.230403,"redis_calls":10,"redis_duration_s":0.00461,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002265,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001371,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0297,"completed_at":"2026-07-04T13:33:17.996Z","load_balancing_strategy":"primary","db_duration_s":0.00557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2360vm-test-run-nixbot-gitlab> gitlab # [  223.322229] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:17.999Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ed2ff6fd8c11d63a51530fb5","created_at":"2026-07-04T13:33:17.737Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.738Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ed2ff6fd8c11d63a51530fb5: start","job_status":"start","queue_duration_s":0.26029,"scheduling_latency_s":0.260315}2361vm-test-run-nixbot-gitlab> gitlab # [  223.323759] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ed2ff6fd8c11d63a51530fb5","created_at":"2026-07-04T13:33:17.737Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.738Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ed2ff6fd8c11d63a51530fb5: done: 0.025505 sec","job_status":"done","queue_duration_s":0.26029,"scheduling_latency_s":0.260315,"redis_calls":7,"redis_duration_s":0.00305,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025505,"completed_at":"2026-07-04T13:33:18.024Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2362vm-test-run-nixbot-gitlab> gitlab # [  223.382211] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.026Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4ecd759b8fb04cf6da8b8113","created_at":"2026-07-04T13:33:17.739Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.741Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4ecd759b8fb04cf6da8b8113: start","job_status":"start","queue_duration_s":0.284817,"scheduling_latency_s":0.284839}2363vm-test-run-nixbot-gitlab> gitlab # [  223.408415] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4ecd759b8fb04cf6da8b8113","created_at":"2026-07-04T13:33:17.739Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.741Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4ecd759b8fb04cf6da8b8113: done: 0.025816 sec","job_status":"done","queue_duration_s":0.284817,"scheduling_latency_s":0.284839,"redis_calls":8,"redis_duration_s":0.004278,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00343,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025816,"completed_at":"2026-07-04T13:33:18.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2364vm-test-run-nixbot-gitlab> gitlab # [  223.480569] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.054Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9787f566a9ec192884f48c23","created_at":"2026-07-04T13:33:17.742Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.744Z","job_size_bytes":66,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9787f566a9ec192884f48c23: start","job_status":"start","queue_duration_s":0.310542,"scheduling_latency_s":0.310566}2365vm-test-run-nixbot-gitlab> gitlab # [  223.502079] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.074Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9787f566a9ec192884f48c23","created_at":"2026-07-04T13:33:17.742Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.744Z","job_size_bytes":66,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9787f566a9ec192884f48c23: done: 0.020194 sec","job_status":"done","queue_duration_s":0.310542,"scheduling_latency_s":0.310566,"redis_calls":7,"redis_duration_s":0.0034029999999999998,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020194,"completed_at":"2026-07-04T13:33:18.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2366vm-test-run-nixbot-gitlab> gitlab # [  223.623299] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"55cfc29fce8fce109108c497","created_at":"2026-07-04T13:33:17.745Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.746Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-55cfc29fce8fce109108c497: start","job_status":"start","queue_duration_s":0.330334,"scheduling_latency_s":0.330356}2367vm-test-run-nixbot-gitlab> gitlab # [  223.624822] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"55cfc29fce8fce109108c497","created_at":"2026-07-04T13:33:17.745Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.746Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-55cfc29fce8fce109108c497: done: 0.030195 sec","job_status":"done","queue_duration_s":0.330334,"scheduling_latency_s":0.330356,"redis_calls":7,"redis_duration_s":0.003081,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000833,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030195,"completed_at":"2026-07-04T13:33:18.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2368vm-test-run-nixbot-gitlab> gitlab # [  223.699359] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.109Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"fb57ae834af0908ad6833328","created_at":"2026-07-04T13:33:17.798Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.800Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fb57ae834af0908ad6833328: start","job_status":"start","queue_duration_s":0.309075,"scheduling_latency_s":0.309098}2369vm-test-run-nixbot-gitlab> gitlab # [  223.700410] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.133Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"fb57ae834af0908ad6833328","created_at":"2026-07-04T13:33:17.798Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.800Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-fb57ae834af0908ad6833328: done: 0.023932 sec","job_status":"done","queue_duration_s":0.309075,"scheduling_latency_s":0.309098,"redis_calls":7,"redis_duration_s":0.0032500000000000003,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000795,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023932,"completed_at":"2026-07-04T13:33:18.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2370vm-test-run-nixbot-gitlab> gitlab # [  223.756804] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.136Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91d6ba9486d5893241db76fd","created_at":"2026-07-04T13:33:17.801Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.803Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91d6ba9486d5893241db76fd: start","job_status":"start","queue_duration_s":0.33365,"scheduling_latency_s":0.333725}2371vm-test-run-nixbot-gitlab> gitlab # [  223.782426] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.165Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91d6ba9486d5893241db76fd","created_at":"2026-07-04T13:33:17.801Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.803Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91d6ba9486d5893241db76fd: done: 0.028637 sec","job_status":"done","queue_duration_s":0.33365,"scheduling_latency_s":0.333725,"redis_calls":7,"redis_duration_s":0.002436,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000672,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028637,"completed_at":"2026-07-04T13:33:18.165Z","load_balancing_strategy":"primary","db_duration_s":0.000815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2372vm-test-run-nixbot-gitlab> gitlab # [  223.912438] gitaly[1387]: time="2026-07-04T13:33:18.223Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAREN2K7ZR4518TXSHCN8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:48.219" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:18.221" grpc.time_ms=1.786 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2373vm-test-run-nixbot-gitlab> gitlab # [  223.914166] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 872374vm-test-run-nixbot-gitlab> gitlab # [  223.914943] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.168Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"8f3acf5c1b1d6ae8af10bda1","created_at":"2026-07-04T13:33:17.803Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.807Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-8f3acf5c1b1d6ae8af10bda1: start","job_status":"start","queue_duration_s":0.360971,"scheduling_latency_s":0.360999}2375vm-test-run-nixbot-gitlab> gitlab # [  223.916072] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.215Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"8f3acf5c1b1d6ae8af10bda1","created_at":"2026-07-04T13:33:17.803Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.807Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-8f3acf5c1b1d6ae8af10bda1: done: 0.046381 sec","job_status":"done","queue_duration_s":0.360971,"scheduling_latency_s":0.360999,"redis_calls":10,"redis_duration_s":0.006794,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002416,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000440558,"duration_s":0.046381,"completed_at":"2026-07-04T13:33:18.215Z","load_balancing_strategy":"primary","db_duration_s":0.007957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2376vm-test-run-nixbot-gitlab> gitlab # [  224.012817] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.217Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8328bc36128085c12dedf31a","created_at":"2026-07-04T13:33:17.850Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.852Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8328bc36128085c12dedf31a: start","job_status":"start","queue_duration_s":0.364994,"scheduling_latency_s":0.365015}2377vm-test-run-nixbot-gitlab> gitlab # [  224.014193] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.350Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8328bc36128085c12dedf31a","created_at":"2026-07-04T13:33:17.850Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-a5a93fe305074957","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:17.852Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8328bc36128085c12dedf31a: done: 0.133338 sec","job_status":"done","queue_duration_s":0.364994,"scheduling_latency_s":0.365015,"gitaly_calls":3,"gitaly_duration_s":0.009675,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.012033,"redis_read_bytes":98,"redis_write_bytes":6011,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002179,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00205,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000315,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002402,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00106,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000462,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09188,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.107731849,"duration_s":0.133338,"completed_at":"2026-07-04T13:33:18.350Z","load_balancing_strategy":"primary","db_duration_s":0.029374,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2378vm-test-run-nixbot-gitlab> gitlab # [  224.071911] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"81c8a823fd0d6a4901fc8d9d","created_at":"2026-07-04T13:33:18.273Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-02378bcc77c14f1f","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:18.275Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-81c8a823fd0d6a4901fc8d9d: start","job_status":"start","queue_duration_s":0.076647,"scheduling_latency_s":0.076677}2379vm-test-run-nixbot-gitlab> gitlab # [  224.098194] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:18.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"81c8a823fd0d6a4901fc8d9d","created_at":"2026-07-04T13:33:18.273Z","trace_propagation_headers":{"sentry-trace":"6516efb673174bab9662e0c56eee618a-02378bcc77c14f1f","baggage":"sentry-trace_id=6516efb673174bab9662e0c56eee618a,sentry-release=Unknown"},"correlation_id":"01KWPNAQFRQ65YB4WTQTBENE2K","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:18.275Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-81c8a823fd0d6a4901fc8d9d: done: 0.024774 sec","job_status":"done","queue_duration_s":0.076647,"scheduling_latency_s":0.076677,"redis_calls":7,"redis_duration_s":0.001739,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024774,"completed_at":"2026-07-04T13:33:18.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2380vm-test-run-nixbot-gitlab> gitlab # [  224.201331] gitaly[1387]: time="2026-07-04T13:33:18.285Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:48.284" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:18.284" grpc.time_ms=0.968 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2381vm-test-run-nixbot-gitlab> gitlab # [  224.202792] gitaly[1387]: time="2026-07-04T13:33:18.308Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:48.307" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:18.307" grpc.time_ms=0.753 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2382vm-test-run-nixbot-gitlab> gitlab # [  224.204263] gitaly[1387]: time="2026-07-04T13:33:18.339Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAQFRQ65YB4WTQTBENE2K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:48.338" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:18.338" grpc.time_ms=0.817 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2383vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 1.20 seconds)2384vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': '2026-07-04T13:33:17.410Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T13:33:17.614Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2385vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2386vm-test-run-nixbot-gitlab> gitlab # [  225.500455] gitaly[1387]: time="2026-07-04T13:33:20.511Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNATKCYN512K4QZD9AKNN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:50.510" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:20.510" grpc.time_ms=0.67 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2387vm-test-run-nixbot-gitlab> gitlab # [  225.593317] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 2392388vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.30 seconds)2389vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': '2026-07-04T13:33:17.410Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T13:33:17.614Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2390vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2391vm-test-run-nixbot-gitlab> gitlab # [  226.723005] gitaly[1387]: time="2026-07-04T13:33:21.734Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAVVXGPG0GNQKB45A7TYS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:51.533" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:21.733" grpc.time_ms=0.657 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2392vm-test-run-nixbot-gitlab> gitlab # [  226.747389] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 972393vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.15 seconds)2394vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': '2026-07-04T13:33:17.410Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-04T13:33:17.614Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2395vm-test-run-nixbot-gitlab> gitlab # [  226.807252] nix-daemon[3393]: accepted connection from pid 3672, user nixbot2396vm-test-run-nixbot-gitlab> gitlab # [  226.885910] gitaly[1387]: time="2026-07-04T13:33:21.897Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:51.695" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:21.896" grpc.time_ms=1.389 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2397vm-test-run-nixbot-gitlab> gitlab # [  226.997528] gitaly[1387]: time="2026-07-04T13:33:22.009Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T13:33:31.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:21.994" grpc.time_ms=14.75 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2398vm-test-run-nixbot-gitlab> gitlab # [  227.062983] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.074Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a9b033342f0d3ab5e76504bf","created_at":"2026-07-04T13:33:22.068Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-adfe84e8770143df","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.071Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a9b033342f0d3ab5e76504bf: start","job_status":"start","queue_duration_s":0.002078,"scheduling_latency_s":0.002117}2399vm-test-run-nixbot-gitlab> gitlab # [  227.099485] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2492400vm-test-run-nixbot-gitlab> gitlab # [  227.100613] nixbot[3145]: {"ts": "2026-07-04T13:33:22.100Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fe2a76c9ff364b509bf59bd688e036c631d88243 \"HTTP/1.1 201 Created\""}2401vm-test-run-nixbot-gitlab> gitlab # [  227.173131] gitaly[1387]: time="2026-07-04T13:33:22.184Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:22.183" grpc.time_ms=0.522 pid=1387 span.kind=server system=grpc2402vm-test-run-nixbot-gitlab> gitlab # [  227.223109] gitaly[1387]: time="2026-07-04T13:33:22.235Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:22.234" grpc.time_ms=0.382 pid=1387 span.kind=server system=grpc2403vm-test-run-nixbot-gitlab> gitlab # [  227.265585] gitaly[1387]: time="2026-07-04T13:33:22.277Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T13:33:22.276" grpc.time_ms=1.053 pid=1387 span.kind=server system=grpc2404vm-test-run-nixbot-gitlab> gitlab # [  227.279200] nix-daemon[3393]: accepted connection from pid 3706, user nixbot2405vm-test-run-nixbot-gitlab> gitlab # [  227.285654] gitaly[1387]: time="2026-07-04T13:33:22.297Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" operations="[repack_refs]" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2406vm-test-run-nixbot-gitlab> gitlab # [  227.288649] gitaly[1387]: time="2026-07-04T13:33:22.300Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" pid=1387 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-04 13:32:54.385805017 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2407vm-test-run-nixbot-gitlab> gitlab # [  227.304392] gitaly[1387]: time="2026-07-04T13:33:22.316Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" pid=1387 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2408vm-test-run-nixbot-gitlab> gitlab # [  227.305448] gitaly[1387]: time="2026-07-04T13:33:22.316Z" level=info msg="optimized repository" clean_stale_data_time=14.789869ms clean_work_trees_time="93.029µs" commit_graph_time="8.381µs" correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.661µs" pid=1387 prune_time="8.101µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=15.121754ms2409vm-test-run-nixbot-gitlab> gitlab # [  227.305940] gitaly[1387]: time="2026-07-04T13:33:22.316Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2410vm-test-run-nixbot-gitlab> gitlab # [  227.372417] gitaly[1387]: time="2026-07-04T13:33:22.384Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ diskcache=bcd56149-9240-49ae-b56f-cd98ca505a90 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2411vm-test-run-nixbot-gitlab> gitlab # [  227.373405] gitaly[1387]: time="2026-07-04T13:33:22.384Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=132 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8647 command.oublock=0 command.real_time_ms=150 command.spawn_token_fork_ms=0 command.system_time_ms=82 command.user_time_ms=50 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T13:33:52.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T13:33:22.156" grpc.time_ms=227.775 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2412vm-test-run-nixbot-gitlab> gitlab # [  227.423814] gitaly[1387]: time="2026-07-04T13:33:22.435Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T13:33:52.034" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:22.434" grpc.time_ms=0.8 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2413vm-test-run-nixbot-gitlab> gitlab # [  227.466655] gitaly[1387]: time="2026-07-04T13:33:22.478Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:52.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:22.477" grpc.time_ms=0.627 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2414vm-test-run-nixbot-gitlab> gitlab # [  227.492593] gitaly[1387]: time="2026-07-04T13:33:22.503Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPNAWKMZ91EBJ8WK5VRERH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:22.474" grpc.time_ms=29.007 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2415vm-test-run-nixbot-gitlab> gitlab # [  227.493653] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:22 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 842416vm-test-run-nixbot-gitlab> gitlab # [  227.498304] gitaly[1387]: time="2026-07-04T13:33:22.510Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAW1SK4BE3ZKASJ7RX6EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:52.109" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:22.509" grpc.time_ms=0.71 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2417vm-test-run-nixbot-gitlab> gitlab # [  227.516345] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.528Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a9b033342f0d3ab5e76504bf","created_at":"2026-07-04T13:33:22.068Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-adfe84e8770143df","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.071Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a9b033342f0d3ab5e76504bf: done: 0.454048 sec","job_status":"done","queue_duration_s":0.002078,"scheduling_latency_s":0.002117,"gitaly_calls":4,"gitaly_duration_s":0.245066,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.023072,"redis_read_bytes":173,"redis_write_bytes":20926,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002108,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004202,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.003608,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13379,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.007176,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000465,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000665,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00274,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.016,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126358,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.421540727,"duration_s":0.454048,"completed_at":"2026-07-04T13:33:22.527Z","load_balancing_strategy":"primary","db_duration_s":0.063087,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2418vm-test-run-nixbot-gitlab> gitlab # [  227.589434] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"81ef9530ab5a8fe774b9c01b","created_at":"2026-07-04T13:33:22.148Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.150Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-81ef9530ab5a8fe774b9c01b: start","job_status":"start","queue_duration_s":0.382463,"scheduling_latency_s":0.382487}2419vm-test-run-nixbot-gitlab> gitlab # [  227.589947] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.567Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"81ef9530ab5a8fe774b9c01b","created_at":"2026-07-04T13:33:22.148Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.150Z","job_size_bytes":23,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-81ef9530ab5a8fe774b9c01b: done: 0.035128 sec","job_status":"done","queue_duration_s":0.382463,"scheduling_latency_s":0.382487,"redis_calls":7,"redis_duration_s":0.004586,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035128,"completed_at":"2026-07-04T13:33:22.567Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004433,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2420vm-test-run-nixbot-gitlab> gitlab # [  227.706427] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.569Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1b6a9ce833775497677dccac","created_at":"2026-07-04T13:33:22.394Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.397Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1b6a9ce833775497677dccac: start","job_status":"start","queue_duration_s":0.171836,"scheduling_latency_s":0.171858}2421vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses2422vm-test-run-nixbot-gitlab> gitlab # [  227.707889] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.588Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1b6a9ce833775497677dccac","created_at":"2026-07-04T13:33:22.394Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.397Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1b6a9ce833775497677dccac: done: 0.018693 sec","job_status":"done","queue_duration_s":0.171836,"scheduling_latency_s":0.171858,"redis_calls":7,"redis_duration_s":0.002202,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018693,"completed_at":"2026-07-04T13:33:22.588Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2423vm-test-run-nixbot-gitlab> gitlab # [  227.736868] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.589Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"51821dd8e486655445c48d40","created_at":"2026-07-04T13:33:22.406Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.409Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-51821dd8e486655445c48d40: start","job_status":"start","queue_duration_s":0.179931,"scheduling_latency_s":0.179952}2424vm-test-run-nixbot-gitlab> gitlab # [  227.762776] gitlab-workhorse[1288]: time="2026-07-04T13:33:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672425vm-test-run-nixbot-gitlab> gitlab # [  227.765172] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:22 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 902426vm-test-run-nixbot-gitlab> gitlab # [  227.768632] gitaly[1387]: time="2026-07-04T13:33:22.595Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAWPB0HBSCGB2493SPGT5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:22.579" pid=1387 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc2427vm-test-run-nixbot-gitlab> gitlab # [  227.837562] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.661Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"51821dd8e486655445c48d40","created_at":"2026-07-04T13:33:22.406Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.409Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-51821dd8e486655445c48d40: done: 0.071767 sec","job_status":"done","queue_duration_s":0.179931,"scheduling_latency_s":0.179952,"redis_calls":7,"redis_duration_s":0.002454,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.051,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.071767,"completed_at":"2026-07-04T13:33:22.661Z","load_balancing_strategy":"primary","db_duration_s":0.051181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2428vm-test-run-nixbot-gitlab> gitlab # [  227.850233] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.662Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0bbaaa5f17a4a71a6bc104f2","created_at":"2026-07-04T13:33:22.416Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.419Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0bbaaa5f17a4a71a6bc104f2: start","job_status":"start","queue_duration_s":0.242893,"scheduling_latency_s":0.242918}2429vm-test-run-nixbot-gitlab> gitlab # [  227.875109] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0bbaaa5f17a4a71a6bc104f2","created_at":"2026-07-04T13:33:22.416Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.419Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0bbaaa5f17a4a71a6bc104f2: done: 0.021938 sec","job_status":"done","queue_duration_s":0.242893,"scheduling_latency_s":0.242918,"redis_calls":10,"redis_duration_s":0.002369,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000684,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021938,"completed_at":"2026-07-04T13:33:22.684Z","load_balancing_strategy":"primary","db_duration_s":0.002503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2430vm-test-run-nixbot-gitlab> gitlab # [  227.986585] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.686Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5e5c4cb220d7961cfd32adb7","created_at":"2026-07-04T13:33:22.420Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.421Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5e5c4cb220d7961cfd32adb7: start","job_status":"start","queue_duration_s":0.264202,"scheduling_latency_s":0.264223}2431vm-test-run-nixbot-gitlab> gitlab # [  227.988105] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.703Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5e5c4cb220d7961cfd32adb7","created_at":"2026-07-04T13:33:22.420Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.421Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5e5c4cb220d7961cfd32adb7: done: 0.01755 sec","job_status":"done","queue_duration_s":0.264202,"scheduling_latency_s":0.264223,"redis_calls":7,"redis_duration_s":0.001641,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01755,"completed_at":"2026-07-04T13:33:22.703Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2432vm-test-run-nixbot-gitlab> gitlab # [  228.056413] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.704Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6d1a1f29fd65fd0893966083","created_at":"2026-07-04T13:33:22.422Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.424Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6d1a1f29fd65fd0893966083: start","job_status":"start","queue_duration_s":0.280595,"scheduling_latency_s":0.280616}2433vm-test-run-nixbot-gitlab> gitlab # [  228.062228] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.723Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6d1a1f29fd65fd0893966083","created_at":"2026-07-04T13:33:22.422Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.424Z","job_size_bytes":19,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6d1a1f29fd65fd0893966083: done: 0.01869 sec","job_status":"done","queue_duration_s":0.280595,"scheduling_latency_s":0.280616,"redis_calls":8,"redis_duration_s":0.001952,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00147,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01869,"completed_at":"2026-07-04T13:33:22.723Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2434vm-test-run-nixbot-gitlab> gitlab # [  228.138485] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.724Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"cb80ca663678717d1462d7ed","created_at":"2026-07-04T13:33:22.425Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.426Z","job_size_bytes":66,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-cb80ca663678717d1462d7ed: start","job_status":"start","queue_duration_s":0.298283,"scheduling_latency_s":0.298303}2435vm-test-run-nixbot-gitlab> gitlab # [  228.156149] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.747Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"cb80ca663678717d1462d7ed","created_at":"2026-07-04T13:33:22.425Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.426Z","job_size_bytes":66,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-cb80ca663678717d1462d7ed: done: 0.0222 sec","job_status":"done","queue_duration_s":0.298283,"scheduling_latency_s":0.298303,"redis_calls":7,"redis_duration_s":0.006109000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0222,"completed_at":"2026-07-04T13:33:22.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2436vm-test-run-nixbot-gitlab> gitlab # [  228.281460] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.748Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d25ea5f2642117fd5e42949f","created_at":"2026-07-04T13:33:22.427Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.428Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d25ea5f2642117fd5e42949f: start","job_status":"start","queue_duration_s":0.319677,"scheduling_latency_s":0.319698}2437vm-test-run-nixbot-gitlab> gitlab # [  228.282704] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.772Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d25ea5f2642117fd5e42949f","created_at":"2026-07-04T13:33:22.427Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.428Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d25ea5f2642117fd5e42949f: done: 0.024352 sec","job_status":"done","queue_duration_s":0.319677,"scheduling_latency_s":0.319698,"redis_calls":7,"redis_duration_s":0.001733,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024352,"completed_at":"2026-07-04T13:33:22.772Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2438vm-test-run-nixbot-gitlab> gitlab # [  228.318348] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.774Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"59300a60ea08b8ab89e947a9","created_at":"2026-07-04T13:33:22.483Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.484Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-59300a60ea08b8ab89e947a9: start","job_status":"start","queue_duration_s":0.289668,"scheduling_latency_s":0.289689}2439vm-test-run-nixbot-gitlab> gitlab # [  228.344808] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.792Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"59300a60ea08b8ab89e947a9","created_at":"2026-07-04T13:33:22.483Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.484Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-59300a60ea08b8ab89e947a9: done: 0.01854 sec","job_status":"done","queue_duration_s":0.289668,"scheduling_latency_s":0.289689,"redis_calls":7,"redis_duration_s":0.0018739999999999998,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000739,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01854,"completed_at":"2026-07-04T13:33:22.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2440vm-test-run-nixbot-gitlab> gitlab # [  228.426180] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.794Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a34c96160be73dad010482db","created_at":"2026-07-04T13:33:22.484Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.486Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a34c96160be73dad010482db: start","job_status":"start","queue_duration_s":0.307995,"scheduling_latency_s":0.308015}2441vm-test-run-nixbot-gitlab> gitlab # [  228.436105] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.809Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a34c96160be73dad010482db","created_at":"2026-07-04T13:33:22.484Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.486Z","job_size_bytes":6,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a34c96160be73dad010482db: done: 0.015884 sec","job_status":"done","queue_duration_s":0.307995,"scheduling_latency_s":0.308015,"redis_calls":7,"redis_duration_s":0.0018340000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015884,"completed_at":"2026-07-04T13:33:22.809Z","load_balancing_strategy":"primary","db_duration_s":0.00039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2442vm-test-run-nixbot-gitlab> gitlab # [  228.570426] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.811Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"ec5c5ce1382639dc4938810c","created_at":"2026-07-04T13:33:22.486Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.487Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ec5c5ce1382639dc4938810c: start","job_status":"start","queue_duration_s":0.323836,"scheduling_latency_s":0.323862}2443vm-test-run-nixbot-gitlab> gitlab # [  228.572116] sidekiq-cluster[2137]: {"severity":"INFO","time":"2026-07-04T13:33:22.829Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"ec5c5ce1382639dc4938810c","created_at":"2026-07-04T13:33:22.486Z","trace_propagation_headers":{"sentry-trace":"b972864c190d4c35a97a58a63993311a-d08339774c7b4fd6","baggage":"sentry-trace_id=b972864c190d4c35a97a58a63993311a,sentry-release=Unknown"},"correlation_id":"01KWPNAW1SK4BE3ZKASJ7RX6EZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-04T13:33:22.487Z","job_size_bytes":3,"pid":2137,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ec5c5ce1382639dc4938810c: done: 0.018141 sec","job_status":"done","queue_duration_s":0.323836,"scheduling_latency_s":0.323862,"redis_calls":7,"redis_duration_s":0.001835,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018141,"completed_at":"2026-07-04T13:33:22.829Z","load_balancing_strategy":"primary","db_duration_s":0.001935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2444vm-test-run-nixbot-gitlab> gitlab # [  228.596565] nix-daemon[3393]: accepted connection from pid 3746, user nixbot2445vm-test-run-nixbot-gitlab> gitlab # [  228.598352] gitaly[1387]: time="2026-07-04T13:33:22.596Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAWPB0HBSCGB2493SPGT5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:22.579" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2446vm-test-run-nixbot-gitlab> gitlab # [  228.618219] gitaly[1387]: time="2026-07-04T13:33:22.597Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAWPB0HBSCGB2493SPGT5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T13:33:22.579" grpc.time_ms=17.405 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc2447vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses, in 0.96 seconds)2448vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T13:33:07.619Z', 'started_at': None, 'finished_at': '2026-07-04T13:33:17.410Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'fe2a76c9ff364b509bf59bd688e036c631d88243', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-04T13:33:17.614Z', 'started_at': None, 'finished_at': '2026-07-04T13:33:22.058Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2449vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 15.88 seconds)2450vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 263.37 seconds)2451vm-test-run-nixbot-gitlab> gitlab # [  228.687125] gitaly[1387]: time="2026-07-04T13:33:23.699Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPNAXPN5EAB3FPAYCF8D7NB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T13:33:53.497" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T13:33:23.698" grpc.time_ms=0.711 pid=1387 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2452vm-test-run-nixbot-gitlab> gitlab # [  228.708532] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/07/04:13:33:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fe2a76c9ff364b509bf59bd688e036c631d88243/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 1782453vm-test-run-nixbot-gitlab> test script finished in 263.40s2454vm-test-run-nixbot-gitlab> cleanup2455vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2456vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13)2457vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.15 seconds)24582459post-build step Upload coverage to codecov: ok2460Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab