these 13 derivations will be built: /nix/store/z1glgcnqwaqx2cbii869clr98f0w4bnw-nixbot-effects.drv /nix/store/rf2aygjiijp0cpi8czq3kz7b8asnvjrf-unit-nixbot.service.drv /nix/store/k0dm7yzsnsj18rf2qbl841i6jd4684q5-system-units.drv /nix/store/ls1lahz5z553cz5v1ycxdl0v6y314k38-etc.drv /nix/store/ww4jp0hwx2n1yqywg21sd8laan3bngi3-activate.drv /nix/store/7bg5i98c7xj6wxd70aww4wax0mqmi3i2-nixos-system-gitlab-test.drv /nix/store/9zhm3aqbmkf7ac0n1y4pwxxb1syh24mh-closure-info.drv /nix/store/y2kdxcqjrrfz1gsq6i510qhsf6vfhmfb-closure-info.drv /nix/store/amj7l5k7chy2ihclpz35aamhmd058ypq-run-nixos-vm.drv /nix/store/l88zq2dmqc8kjvwy1xwdmay7v98n7gv2-nixos-vm.drv /nix/store/ak2cb3vfp3ghmfdx7x4lyqw6phz81rsj-driverConfiguration.json.drv /nix/store/dq1gk4mk0j6w4rwmg2mxhj6s0qs4p185-nixos-test-driver-nixbot-gitlab.drv /nix/store/qf61q619c8j8l6cb6pp54c4c4w7gfk1v-vm-test-run-nixbot-gitlab.drv building '/nix/store/z1glgcnqwaqx2cbii869clr98f0w4bnw-nixbot-effects.drv' on 'ssh-ng://nix@jamie' building '/nix/store/z1glgcnqwaqx2cbii869clr98f0w4bnw-nixbot-effects.drv' nixbot-effects> Sourcing python-remove-tests-dir-hook nixbot-effects> Sourcing python-catch-conflicts-hook.sh nixbot-effects> Sourcing python-remove-bin-bytecode-hook.sh nixbot-effects> Sourcing pypa-build-hook nixbot-effects> Using pypaBuildPhase nixbot-effects> Sourcing python-runtime-deps-check-hook nixbot-effects> Using pythonRuntimeDepsCheckHook nixbot-effects> Sourcing pypa-install-hook nixbot-effects> Using pypaInstallPhase nixbot-effects> Sourcing python-imports-check-hook.sh nixbot-effects> Using pythonImportsCheckPhase nixbot-effects> Sourcing python-namespaces-hook nixbot-effects> Sourcing python-catch-conflicts-hook.sh nixbot-effects> Running phase: unpackPhase nixbot-effects> unpacking source archive /nix/store/mhkb6cayhm3b47y4n71h82annqs0g777-nixbot_effects nixbot-effects> source root is nixbot_effects nixbot-effects> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot_effects/pyproject.toml" nixbot-effects> Running phase: patchPhase nixbot-effects> Running phase: updateAutotoolsGnuConfigScriptsPhase nixbot-effects> Running phase: configurePhase nixbot-effects> no configure script, doing nothing nixbot-effects> Running phase: buildPhase nixbot-effects> Executing pypaBuildPhase nixbot-effects> Creating a wheel... nixbot-effects> pypa build flags: --no-isolation --outdir dist/ --wheel nixbot-effects> * Getting build dependencies for wheel... nixbot-effects> * Building wheel... nixbot-effects> Successfully built nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished creating a wheel... nixbot-effects> Finished executing pypaBuildPhase nixbot-effects> Running phase: pythonRuntimeDepsCheckHook nixbot-effects> Executing pythonRuntimeDepsCheck nixbot-effects> Checking runtime dependencies for nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished executing pythonRuntimeDepsCheck nixbot-effects> Running phase: installPhase nixbot-effects> Executing pypaInstallPhase nixbot-effects> Successfully installed nixbot_effects-0.0.1-py3-none-any.whl nixbot-effects> Finished executing pypaInstallPhase nixbot-effects> Running phase: pythonOutputDistPhase nixbot-effects> Executing pythonOutputDistPhase nixbot-effects> Finished executing pythonOutputDistPhase nixbot-effects> Running phase: fixupPhase nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects nixbot-effects> checking for references to /build/ in /nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects... nixbot-effects> patching script interpreter paths in /nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects nixbot-effects> stripping (with command strip and flags -S -p) in /nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects/lib /nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects/bin nixbot-effects> shrinking RPATHs of ELF executables and libraries in /nix/store/jnpni5q33gffr2s3zcmf306fhc75jlb6-nixbot-effects-dist nixbot-effects> checking for references to /build/ in /nix/store/jnpni5q33gffr2s3zcmf306fhc75jlb6-nixbot-effects-dist... nixbot-effects> patching script interpreter paths in /nix/store/jnpni5q33gffr2s3zcmf306fhc75jlb6-nixbot-effects-dist nixbot-effects> Rewriting #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13 to #!/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13 nixbot-effects> wrapping `/nix/store/yzmi9qprp8szbaywlwmn90crmznzkiac-nixbot-effects/bin/nixbot-effects'... nixbot-effects> Executing pythonRemoveTestsDir nixbot-effects> Finished executing pythonRemoveTestsDir nixbot-effects> Running phase: installCheckPhase nixbot-effects> no Makefile or custom installCheckPhase, doing nothing nixbot-effects> Running phase: pythonCatchConflictsPhase nixbot-effects> Running phase: pythonRemoveBinBytecodePhase nixbot-effects> Running phase: pythonImportsCheckPhase nixbot-effects> Executing pythonImportsCheckPhase building '/nix/store/rf2aygjiijp0cpi8czq3kz7b8asnvjrf-unit-nixbot.service.drv' unit-nixbot.service> tribuchet: building on jamie building '/nix/store/k0dm7yzsnsj18rf2qbl841i6jd4684q5-system-units.drv' system-units> tribuchet: building on jamie building '/nix/store/ls1lahz5z553cz5v1ycxdl0v6y314k38-etc.drv' etc> tribuchet: building on jamie building '/nix/store/ww4jp0hwx2n1yqywg21sd8laan3bngi3-activate.drv' activate> tribuchet: building on jamie building '/nix/store/7bg5i98c7xj6wxd70aww4wax0mqmi3i2-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> tribuchet: building on jamie nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/9zhm3aqbmkf7ac0n1y4pwxxb1syh24mh-closure-info.drv' closure-info> tribuchet: building on jamie closure-info> structuredAttrs is enabled building '/nix/store/y2kdxcqjrrfz1gsq6i510qhsf6vfhmfb-closure-info.drv' closure-info> tribuchet: building on jamie closure-info> structuredAttrs is enabled building '/nix/store/amj7l5k7chy2ihclpz35aamhmd058ypq-run-nixos-vm.drv' run-nixos-vm> tribuchet: building on jamie building '/nix/store/l88zq2dmqc8kjvwy1xwdmay7v98n7gv2-nixos-vm.drv' nixos-vm> tribuchet: building on jamie building '/nix/store/ak2cb3vfp3ghmfdx7x4lyqw6phz81rsj-driverConfiguration.json.drv' driverConfiguration.json> tribuchet: building on jamie driverConfiguration.json> structuredAttrs is enabled building '/nix/store/dq1gk4mk0j6w4rwmg2mxhj6s0qs4p185-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> tribuchet: building on jamie nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! building '/nix/store/qf61q619c8j8l6cb6pp54c4c4w7gfk1v-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie' building '/nix/store/qf61q619c8j8l6cb6pp54c4c4w7gfk1v-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13) vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.0GEY5EaIok', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025) vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: d543501f-021d-40d5-8e46-3def9523486a vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> (finished: start all VMs, in 38.70 seconds) vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Booting from ROM... vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7qa8dqqf3gbhim1qygmin3vzgy8xghyy-nixos-system-gitlab-test/init regInfo=/nix/store/lx8h2dkrm7msnkrhnmwi65cg2iqiblc8-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: using sched offset of 471212081 cycles vm-test-run-nixbot-gitlab> gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor vm-test-run-nixbot-gitlab> gitlab # [ 0.000822] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000859] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs vm-test-run-nixbot-gitlab> gitlab # [ 0.000861] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT vm-test-run-nixbot-gitlab> gitlab # [ 0.000913] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.002764] found SMP MP-table at [mem 0x000f5470-0x000f547f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002774] Using GB pages for direct mapping vm-test-run-nixbot-gitlab> gitlab # [ 0.002859] RAMDISK: [mem 0xbe4e0000-0xbffcffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.002866] ACPI: Early table checksum verification disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.002868] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) vm-test-run-nixbot-gitlab> gitlab # [ 0.002871] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002875] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002882] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002884] ACPI: FACS 0x00000000BFFE0000 000040 vm-test-run-nixbot-gitlab> gitlab # [ 0.002886] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002887] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002889] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002890] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] vm-test-run-nixbot-gitlab> gitlab # [ 0.002891] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] vm-test-run-nixbot-gitlab> gitlab # [ 0.002892] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002892] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] vm-test-run-nixbot-gitlab> gitlab # [ 0.002893] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] vm-test-run-nixbot-gitlab> gitlab # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] vm-test-run-nixbot-gitlab> gitlab # [ 0.003377] No NUMA configuration found vm-test-run-nixbot-gitlab> gitlab # [ 0.003378] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.003382] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005701] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.005703] DMA [mem 0x0000000000001000-0x0000000000ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005704] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005705] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005706] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.005707] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.005708] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.005708] node 0: [mem 0x0000000000001000-0x000000000009efff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005709] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005710] node 0: [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005711] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.005731] On node 0, zone DMA: 1 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.005983] On node 0, zone DMA: 97 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.108932] On node 0, zone Normal: 40 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.109960] ACPI: PM-Timer IO Port: 0x608 vm-test-run-nixbot-gitlab> gitlab # [ 0.109978] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) vm-test-run-nixbot-gitlab> gitlab # [ 0.110017] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 vm-test-run-nixbot-gitlab> gitlab # [ 0.110020] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) vm-test-run-nixbot-gitlab> gitlab # [ 0.110022] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.110023] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.110024] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.110025] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.110027] ACPI: Using ACPI (MADT) for SMP configuration information vm-test-run-nixbot-gitlab> gitlab # [ 0.110028] ACPI: HPET id: 0x8086a201 base: 0xfed00000 vm-test-run-nixbot-gitlab> gitlab # [ 0.110034] TSC deadline timer available vm-test-run-nixbot-gitlab> gitlab # [ 0.110039] CPU topo: Max. logical packages: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.110039] CPU topo: Max. logical dies: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.110040] CPU topo: Max. dies per package: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.110043] CPU topo: Max. threads per core: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.110044] CPU topo: Num. cores per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.110044] CPU topo: Num. threads per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.110044] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.110068] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() vm-test-run-nixbot-gitlab> gitlab # [ 0.110085] kvm-guest: KVM setup pv remote TLB flush vm-test-run-nixbot-gitlab> gitlab # [ 0.110089] kvm-guest: setup PV sched yield vm-test-run-nixbot-gitlab> gitlab # [ 0.110108] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.110110] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.110110] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.110112] [mem 0xc0000000-0xfeffbfff] available for PCI devices vm-test-run-nixbot-gitlab> gitlab # [ 0.110113] Booting paravirtualized kernel on KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.110117] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.114552] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 vm-test-run-nixbot-gitlab> gitlab # [ 0.116694] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 vm-test-run-nixbot-gitlab> gitlab # [ 0.116746] kvm-guest: PV spinlocks enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.116748] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.116751] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7qa8dqqf3gbhim1qygmin3vzgy8xghyy-nixos-system-gitlab-test/init regInfo=/nix/store/lx8h2dkrm7msnkrhnmwi65cg2iqiblc8-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.116842] Unknown kernel command line parameters "regInfo=/nix/store/lx8h2dkrm7msnkrhnmwi65cg2iqiblc8-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.116854] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.116855] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.125313] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.129621] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.129692] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.200270] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.200278] Built 1 zonelists, mobility grouping on. Total pages: 1572726 vm-test-run-nixbot-gitlab> gitlab # [ 0.200279] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.202729] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.211688] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.224777] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.234453] ftrace: allocating 48565 entries in 192 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.234456] ftrace: allocated 192 pages with 2 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.235272] Dynamic Preempt: lazy vm-test-run-nixbot-gitlab> gitlab # [ 0.235410] rcu: Preemptible hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.235411] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.235412] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.235413] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.235413] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.235414] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.235414] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.235415] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.235439] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.235440] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.235441] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.239735] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.240009] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.240125] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.247470] Console: colour VGA+ 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.247475] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.291356] printk: legacy console [ttyS0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.490888] ACPI: Core revision 20250807 vm-test-run-nixbot-gitlab> gitlab # [ 0.492454] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.495233] APIC: Switch to symmetric I/O mode setup vm-test-run-nixbot-gitlab> gitlab # [ 0.497004] x2apic enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.498278] APIC: Switched APIC routing to: physical x2apic vm-test-run-nixbot-gitlab> gitlab # [ 0.499986] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() vm-test-run-nixbot-gitlab> gitlab # [ 0.502206] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() vm-test-run-nixbot-gitlab> gitlab # [ 0.504987] kvm-guest: setup PV IPIs vm-test-run-nixbot-gitlab> gitlab # [ 0.507299] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 vm-test-run-nixbot-gitlab> gitlab # [ 0.509128] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.512214] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) vm-test-run-nixbot-gitlab> gitlab # [ 0.514329] x86/cpu: User Mode Instruction Prevention (UMIP) activated vm-test-run-nixbot-gitlab> gitlab # [ 0.515351] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 vm-test-run-nixbot-gitlab> gitlab # [ 0.517212] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.518217] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto vm-test-run-nixbot-gitlab> gitlab # [ 0.520213] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl vm-test-run-nixbot-gitlab> gitlab # [ 0.521213] Transient Scheduler Attacks: Mitigation: Clear CPU buffers vm-test-run-nixbot-gitlab> gitlab # [ 0.522213] Spectre V2 : Mitigation: Enhanced / Automatic IBRS vm-test-run-nixbot-gitlab> gitlab # [ 0.523212] Speculative Return Stack Overflow: Mitigation: Safe RET vm-test-run-nixbot-gitlab> gitlab # [ 0.524212] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization vm-test-run-nixbot-gitlab> gitlab # [ 0.525218] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier vm-test-run-nixbot-gitlab> gitlab # [ 0.527213] active return thunk: srso_alias_return_thunk vm-test-run-nixbot-gitlab> gitlab # [ 0.528241] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.529212] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.530212] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.531212] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' vm-test-run-nixbot-gitlab> gitlab # [ 0.532212] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.534212] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.535212] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.536212] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.538212] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' vm-test-run-nixbot-gitlab> gitlab # [ 0.540213] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 vm-test-run-nixbot-gitlab> gitlab # [ 0.541212] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.542212] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 vm-test-run-nixbot-gitlab> gitlab # [ 0.543212] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 vm-test-run-nixbot-gitlab> gitlab # [ 0.544212] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 vm-test-run-nixbot-gitlab> gitlab # [ 0.546212] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.548212] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 vm-test-run-nixbot-gitlab> gitlab # [ 0.550212] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. vm-test-run-nixbot-gitlab> gitlab # [ 0.580311] Freeing SMP alternatives memory: 44K vm-test-run-nixbot-gitlab> gitlab # [ 0.581215] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.583341] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.585280] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.586213] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.587422] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.589264] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.590365] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.593285] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) vm-test-run-nixbot-gitlab> gitlab # [ 0.594788] Performance Events: Fam17h+ core perfctr, AMD PMU driver. vm-test-run-nixbot-gitlab> gitlab # [ 0.595222] ... version: 2 vm-test-run-nixbot-gitlab> gitlab # [ 0.596213] ... bit width: 48 vm-test-run-nixbot-gitlab> gitlab # [ 0.597214] ... generic counters: 6 vm-test-run-nixbot-gitlab> gitlab # [ 0.598214] ... generic bitmap: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.599214] ... fixed-purpose counters: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.600214] ... fixed-purpose bitmap: 0000000000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.601214] ... value mask: 0000ffffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.602214] ... max period: 00007fffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.603223] ... global_ctrl mask: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.604342] signal: max sigframe size: 3376 vm-test-run-nixbot-gitlab> gitlab # [ 0.605299] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.606218] rcu: Max phase no-delay instances is 400. vm-test-run-nixbot-gitlab> gitlab # [ 0.607402] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.612779] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.613622] smpboot: x86: Booting SMP configuration: vm-test-run-nixbot-gitlab> gitlab # [ 0.614217] .... node #0, CPUs: #1 #2 #3 vm-test-run-nixbot-gitlab> gitlab # [ 0.618308] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.620214] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) vm-test-run-nixbot-gitlab> gitlab # [ 0.621769] Memory: 6013416K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.624213] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.625476] x86/mm: Memory block size: 128MB vm-test-run-nixbot-gitlab> gitlab # [ 0.628395] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.630264] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.631290] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.632328] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.633497] PM: RTC time: 18:01:08, date: 2026-07-01 vm-test-run-nixbot-gitlab> gitlab # [ 0.637084] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.639562] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.640274] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.642258] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.643226] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.644386] audit: type=2000 audit(1782928868.775:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.644489] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.646214] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.647214] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.648214] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.649214] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.650229] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.653432] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 vm-test-run-nixbot-gitlab> gitlab # [ 0.654580] PCI: Using configuration type 1 for base access vm-test-run-nixbot-gitlab> gitlab # [ 0.655215] PCI: Using configuration type 1 for extended access vm-test-run-nixbot-gitlab> gitlab # [ 0.656398] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. vm-test-run-nixbot-gitlab> gitlab # [ 0.658589] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.660214] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.661215] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.662214] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.666499] ACPI: Added _OSI(Module Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.670226] ACPI: Added _OSI(Processor Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.673222] ACPI: Added _OSI(Processor Aggregator Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.683088] ACPI: 1 ACPI AML tables successfully acquired and loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.692237] ACPI: Interpreter enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.693232] ACPI: PM: (supports S0 S3 S4 S5) vm-test-run-nixbot-gitlab> gitlab # [ 0.694214] ACPI: Using IOAPIC for interrupt routing vm-test-run-nixbot-gitlab> gitlab # [ 0.695286] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug vm-test-run-nixbot-gitlab> gitlab # [ 0.701236] PCI: Using E820 reservations for host bridge windows vm-test-run-nixbot-gitlab> gitlab # [ 0.707202] ACPI: Enabled 2 GPEs in block 00 to 0F vm-test-run-nixbot-gitlab> gitlab # [ 0.724184] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) vm-test-run-nixbot-gitlab> gitlab # [ 0.728233] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] vm-test-run-nixbot-gitlab> gitlab # [ 0.736653] acpiphp: Slot [3] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.739364] acpiphp: Slot [4] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.742360] acpiphp: Slot [5] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.745335] acpiphp: Slot [6] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.748370] acpiphp: Slot [7] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.750333] acpiphp: Slot [8] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.753333] acpiphp: Slot [9] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.756338] acpiphp: Slot [10] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.758359] acpiphp: Slot [11] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.761335] acpiphp: Slot [12] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.763270] acpiphp: Slot [13] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.764266] acpiphp: Slot [14] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.766249] acpiphp: Slot [15] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.767603] acpiphp: Slot [16] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.770338] acpiphp: Slot [17] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.774330] acpiphp: Slot [18] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.777308] acpiphp: Slot [19] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.780341] acpiphp: Slot [20] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.783310] acpiphp: Slot [21] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.787325] acpiphp: Slot [22] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.790308] acpiphp: Slot [23] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.793344] acpiphp: Slot [24] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.796308] acpiphp: Slot [25] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.800308] acpiphp: Slot [26] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.803340] acpiphp: Slot [27] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.806391] acpiphp: Slot [28] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.809328] acpiphp: Slot [29] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.813312] acpiphp: Slot [30] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.816328] acpiphp: Slot [31] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.818306] PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.821233] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 0.825221] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.829227] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.833245] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.836216] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.840258] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.845528] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.854818] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.862602] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.874417] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] vm-test-run-nixbot-gitlab> gitlab # [ 0.878386] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.883223] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.886219] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.890220] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.894836] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.901992] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] vm-test-run-nixbot-gitlab> gitlab # [ 0.904703] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.909923] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI vm-test-run-nixbot-gitlab> gitlab # [ 0.916259] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB vm-test-run-nixbot-gitlab> gitlab # [ 0.921409] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.927141] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.948358] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.952275] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.956351] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.960658] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.976266] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 15625 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.982807] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.997258] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] vm-test-run-nixbot-gitlab> gitlab # [ 1.002260] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.006371] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.012367] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.022965] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.034249] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] vm-test-run-nixbot-gitlab> gitlab # [ 1.037254] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.041279] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.045811] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.052257] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] vm-test-run-nixbot-gitlab> gitlab # [ 1.054229] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.055267] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.060267] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.066259] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] vm-test-run-nixbot-gitlab> gitlab # [ 1.067229] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.069268] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.074267] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.079255] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] vm-test-run-nixbot-gitlab> gitlab # [ 1.081229] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.083278] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.087271] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.092581] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.095229] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.096271] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.101279] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.107547] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.111260] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.115356] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.121261] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.130680] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.142289] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.147356] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.158147] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.169289] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] vm-test-run-nixbot-gitlab> gitlab # [ 1.173232] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.175269] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.179314] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.185475] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] vm-test-run-nixbot-gitlab> gitlab # [ 1.187230] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] vm-test-run-nixbot-gitlab> gitlab # [ 1.189300] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.199004] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.200447] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.202426] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.204439] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.206333] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 vm-test-run-nixbot-gitlab> gitlab # [ 1.210473] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 1.211216] iommu: DMA domain TLB invalidation policy: lazy mode vm-test-run-nixbot-gitlab> gitlab # [ 1.213484] ACPI: bus type USB registered vm-test-run-nixbot-gitlab> gitlab # [ 1.215288] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 1.217236] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 1.218243] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 1.221400] NetLabel: Initializing vm-test-run-nixbot-gitlab> gitlab # [ 1.222214] NetLabel: domain hash size = 128 vm-test-run-nixbot-gitlab> gitlab # [ 1.223214] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO vm-test-run-nixbot-gitlab> gitlab # [ 1.225326] NetLabel: unlabeled traffic allowed by default vm-test-run-nixbot-gitlab> gitlab # [ 1.227229] PCI: Using ACPI for IRQ routing vm-test-run-nixbot-gitlab> gitlab # [ 1.229355] pci 0000:00:02.0: vgaarb: setting as boot VGA device vm-test-run-nixbot-gitlab> gitlab # [ 1.230209] pci 0000:00:02.0: vgaarb: bridge control possible vm-test-run-nixbot-gitlab> gitlab # [ 1.230209] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none vm-test-run-nixbot-gitlab> gitlab # [ 1.235220] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 1.236408] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.237215] hpet0: 3 comparators, 64-bit 100.000000 MHz counter vm-test-run-nixbot-gitlab> gitlab # [ 1.243211] clocksource: Switched to clocksource kvm-clock vm-test-run-nixbot-gitlab> gitlab # [ 1.245658] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.247867] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 1.253742] pnp: PnP ACPI init vm-test-run-nixbot-gitlab> gitlab # [ 1.258048] pnp: PnP ACPI: found 6 devices vm-test-run-nixbot-gitlab> gitlab # [ 1.272215] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.278839] clocksource: Switched to clocksource acpi_pm vm-test-run-nixbot-gitlab> gitlab # [ 1.283434] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.289393] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.314786] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.317437] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.319952] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.329476] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.335476] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 1.342380] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.349029] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.354973] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.360955] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.365426] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.369394] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 1.374028] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.377859] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.382143] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.384262] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.388871] pci 0000:00:01.0: PIIX3: Enabling Passive Release vm-test-run-nixbot-gitlab> gitlab # [ 1.393446] pci 0000:00:00.0: Limiting direct PCI/PCI transfers vm-test-run-nixbot-gitlab> gitlab # [ 1.402396] ACPI: \_SB_.LNKD: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.410312] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 11832 usecs vm-test-run-nixbot-gitlab> gitlab # [ 1.415325] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 1.418066] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) vm-test-run-nixbot-gitlab> gitlab # [ 1.418436] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 1.418793] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 1.429385] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.461971] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 1.466147] workingset: timestamp_bits=40 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 1.495287] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 1.496654] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 1.498304] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) vm-test-run-nixbot-gitlab> gitlab # [ 1.503152] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 1.505244] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 1.510860] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.516790] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A vm-test-run-nixbot-gitlab> gitlab # [ 1.526029] Linux agpgart interface v0.103 vm-test-run-nixbot-gitlab> gitlab # [ 1.529562] ACPI: bus type drm_connector registered vm-test-run-nixbot-gitlab> gitlab # [ 1.535780] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 1.537797] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 1.539662] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. vm-test-run-nixbot-gitlab> gitlab # [ 1.539662] Please enable it if your BIOS has the CPPC option. vm-test-run-nixbot-gitlab> gitlab # [ 1.543790] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled vm-test-run-nixbot-gitlab> gitlab # [ 1.546335] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 1.548416] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.551160] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.552476] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.555609] IPI shorthand broadcast: enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.560751] sched_clock: Marking stable (1300030275, 260190552)->(1641891315, -81670488) vm-test-run-nixbot-gitlab> gitlab # [ 1.563688] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.565348] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 1.575807] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 1.577520] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 1.578834] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 1.580512] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 1.582223] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 1.583716] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 1.585520] PM: Magic number: 2:950:37 vm-test-run-nixbot-gitlab> gitlab # [ 1.587860] RAS: Correctable Errors collector initialized. vm-test-run-nixbot-gitlab> gitlab # [ 1.593340] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 1.594670] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 1.724679] Freeing initrd memory: 27584K vm-test-run-nixbot-gitlab> gitlab # [ 1.732691] Freeing unused decrypted memory: 2028K vm-test-run-nixbot-gitlab> gitlab # [ 1.741596] Freeing unused kernel image (initmem) memory: 3644K vm-test-run-nixbot-gitlab> gitlab # [ 1.746545] Write protecting the kernel read-only data: 32768k vm-test-run-nixbot-gitlab> gitlab # [ 1.754434] Freeing unused kernel image (text/rodata gap) memory: 1332K vm-test-run-nixbot-gitlab> gitlab # [ 1.761022] Freeing unused kernel image (rodata/data gap) memory: 816K vm-test-run-nixbot-gitlab> gitlab # [ 1.826300] x86/mm: Checked W+X mappings: passed, no W+X pages found. vm-test-run-nixbot-gitlab> gitlab # [ 1.828352] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 1.844393] systemd[1]: Inserted module 'autofs4' vm-test-run-nixbot-gitlab> gitlab # [ 1.873416] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 1.896568] ACPI: \_SB_.LNKC: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.906845] ACPI: \_SB_.LNKA: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.912313] ACPI: \_SB_.LNKB: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.961630] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 2.302026] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 2.323701] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 2.327743] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 2.331715] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 2.336128] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 2.341451] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 2.422123] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 2.551163] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 2.555443] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 2.561726] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 2.565980] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.570017] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.573347] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.576571] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 2.579533] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.581751] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.583937] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 2.586048] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 2.587907] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.589798] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.591535] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.597384] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 2.609733] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 2.614820] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 2.630432] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 2.634886] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 2.639309] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 2.651022] systemd-journald[152]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 2.656690] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 2.675798] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 2.682353] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 2.686368] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 2.690303] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.694710] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 2.699024] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 2.703507] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 2.708351] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.712368] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 2.718431] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 2.720897] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config vm-test-run-nixbot-gitlab> gitlab # [ 2.726585] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 2.733536] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 2.741492] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 2.745749] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 2.485627] systemd-modules-load[154]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 2.489148] systemd-modules-load[154]: Inserted module 'virtio_balloon' vm-test-run-nixbot-gitlab> gitlab # [ 2.491183] systemd-modules-load[154]: Inserted module 'dm_mod' vm-test-run-nixbot-gitlab> gitlab # [ 2.493141] systemd-modules-load[154]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 2.495088] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 2.497486] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 2.508568] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 2.510374] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.512070] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.513432] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 2.515165] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 2.538805] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 2.544124] systemd-udevd[195]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 2.559907] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 2.599805] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 2.601255] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 2.602751] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 2.967234] uhci_hcd 0000:00:01.2: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 2.968547] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 2.970438] uhci_hcd 0000:00:01.2: detected 2 ports vm-test-run-nixbot-gitlab> gitlab # [ 2.971782] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 vm-test-run-nixbot-gitlab> gitlab # [ 2.973523] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 vm-test-run-nixbot-gitlab> gitlab # [ 2.975469] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 vm-test-run-nixbot-gitlab> gitlab # [ 2.977218] usb usb1: Product: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 2.978859] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 2.980183] usb usb1: SerialNumber: 0000:00:01.2 vm-test-run-nixbot-gitlab> gitlab # [ 2.981934] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 2.982991] hub 1-0:1.0: 2 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 2.984873] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 2.990005] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 2.994454] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 vm-test-run-nixbot-gitlab> gitlab # [ 2.994720] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 2.998753] serio: i8042 KBD port at 0x60,0x64 irq 1 vm-test-run-nixbot-gitlab> gitlab # [ 3.000729] serio: i8042 AUX port at 0x60,0x64 irq 12 vm-test-run-nixbot-gitlab> gitlab # [ 3.004263] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 3.034100] virtio_blk virtio5: [vdb] 11449904 512-byte logical blocks (5.86 GB/5.46 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 3.047346] scsi host0: ata_piix vm-test-run-nixbot-gitlab> gitlab # [ 2.816775] (udev-worker)[224]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on pot[ 3.079572] scsi host1: ata_piix vm-test-run-nixbot-gitlab> gitlab # entially unpredictable interface name.[ 3.081036] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # [ 3.083361] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 2.826529] (udev-worker)[224]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.833241] (udev-worker)[223]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 2.835819] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 2.837777] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 2.839653] systemd[1]: Reach[ 3.099644] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 vm-test-run-nixbot-gitlab> gitlab # ed target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 2.847140] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.849476] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 2.856193] systemd-vconsole-setup[233]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 2.860620] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 2.875457] systemd-fsck[243]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 2.880927] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 3.221384] usb 1-1: new full-speed USB device number 2 using uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 3.244959] ata2: found unknown device (class 0) vm-test-run-nixbot-gitlab> gitlab # [ 3.250353] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 vm-test-run-nixbot-gitlab> gitlab # [ 3.257483] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 vm-test-run-nixbot-gitlab> gitlab # [ 3.330776] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray vm-test-run-nixbot-gitlab> gitlab # [ 3.365662] cdrom: Uniform CD-ROM driver Revision: 3.20 vm-test-run-nixbot-gitlab> gitlab # [ 3.393456] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 vm-test-run-nixbot-gitlab> gitlab # [ 3.398281] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 vm-test-run-nixbot-gitlab> gitlab # [ 3.402762] usb 1-1: Product: QEMU USB Tablet vm-test-run-nixbot-gitlab> gitlab # [ 3.405602] usb 1-1: Manufacturer: QEMU vm-test-run-nixbot-gitlab> gitlab # [ 3.408272] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 vm-test-run-nixbot-gitlab> gitlab # [ 3.446551] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 3.461163] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 3.464655] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 3.476319] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 3.481749] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 3.419413] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 3.858722] EXT4-fs (vda): mounted filesystem d543501f-021d-40d5-8e46-3def9523486a r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 3.602792] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 3.604826] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 3.612640] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.618231] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.632182] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 3.636285] systemd[1]: Mounting /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 3.639894] systemd[1]: Mounting /sysroot/tmp/xchg... vm-test-run-nixbot-gitlab> gitlab # [ 3.643639] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 3.651041] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.653107] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 3.914407] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # [ 3.656755] systemd-sysroot-fstab-check[271]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 3.659689] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.921815] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 3.664047] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.666318] systemd[1]: Reload requested from client PID 271 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 3.669194] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 3.894579] systemd[1]: Reloading finished in 229 ms. vm-test-run-nixbot-gitlab> gitlab # [ 3.951806] systemd-sysroot-fstab-check[271]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.079379] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 4.082911] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 4.086241] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.093345] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.105945] systemd-sysroot-fstab-check[271]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.111910] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.116513] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.122883] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.127233] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 4.131888] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.138232] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.420896] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 4.477702] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 4.481302] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.486419] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 4.492225] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 4.530415] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.535589] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.559557] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.565592] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.573912] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 4.619421] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.623319] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 4.626608] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 4.630249] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.633448] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.638115] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 4.642957] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.646285] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.649490] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 4.653112] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 4.656154] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.659330] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.662571] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.665704] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.669553] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.672526] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 4.678387] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.681719] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.683784] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 4.685608] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.687646] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 4.689277] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.692340] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.694457] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.697275] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 4.699418] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.701208] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.703317] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.705418] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 4.707240] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 4.710794] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.713188] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 4.718487] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.720388] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.725600] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.728643] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.736779] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.739675] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.746553] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.748722] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.755114] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.756930] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.759248] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.761248] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.763570] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.765604] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.767573] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.787183] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.789767] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.793189] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.796281] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 4.945373] initrd-nixos-activation-start[573]: booting system configuration /nix/store/7qa8dqqf3gbhim1qygmin3vzgy8xghyy-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 5.006518] initrd-nixos-activation-start[573]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 5.277430] initrd-nixos-activation-start[596]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 5.366600] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 5.372750] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.385752] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.410705] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 6.036060] systemd-journald[152]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 6.177553] NET: Registered PF_VSOCK protocol family vm-test-run-nixbot-gitlab> gitlab # [ 6.569991] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 6.586466] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 6.589715] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 6.592996] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 6.597180] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 6.741258] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 6.811636] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 6.971814] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 7.083970] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.086407] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.089684] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 7.092999] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 7.095154] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.097456] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.099067] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.101165] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.103149] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 7.104495] systemd[1]: Expecting device /dev/ttyS0... vm-test-run-nixbot-gitlab> gitlab # [ 7.105865] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 7.107433] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.108832] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.110274] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.111865] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 7.113550] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.114842] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.116373] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.117680] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 7.119933] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.122002] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 7.125388] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.127536] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.129281] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.131226] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.133656] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.139274] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.142147] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.145066] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.150084] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.162813] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 7.167108] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 7.173313] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 7.174874] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 7.177392] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.180036] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.185309] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.186811] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.207967] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 7.214699] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 7.222859] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 7.232492] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 7.236659] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.238104] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 7.238582] systemd-journald[792]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 7.249327] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 6.989580] systemd[1]: Queued start job for default target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 6.992954] systemd[1]: Unnecessary job was removed for /dev/vdb. vm-test-run-nixbot-gitlab> gitlab # [ 6.995203] s[ 7.255335] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # ystemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 6.998450] systemd-modules-load[794]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 7.000488] systemd-modules-load[794]: Inserted module 'loop' vm-test-run-nixbot-gitlab> gitlab # [ 7.002386] systemd-modules-load[794]: Inserted module 'tls'[ 7.263304] EXT4-fs (vda): re-mounted d543501f-021d-40d5-8e46-3def9523486a. vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 7.005607] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.007655] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.009453] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.011642] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.013381] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.015387] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 7.017160] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.019139] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 7.020749] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.022260] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 7.023761] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.025526] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.027359] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 7.028662] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 7.030293] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.033238] systemd-oomd[795]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 7.037468] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 7.039394] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 7.304234] systemd-journald[792]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 7.123352] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 7.125595] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.128674] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 7.130670] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.132618] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 7.134493] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 7.135932] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 7.137399] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 7.139277] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 7.141286] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 7.142987] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.144727] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 7.146930] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 7.149215] systemd-udevd[837]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 7.177809] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 7.236117] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 7.289822] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 7.299213] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.328522] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.330460] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.334720] (udev-worker)[890]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.342931] (udev-worker)[883]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 7.347198] (udev-worker)[883]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.361440] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.363910] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.366646] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.370709] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.373719] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.376122] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.661981] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 7.670364] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 7.679259] ACPI: button: Power Button [PWRF] vm-test-run-nixbot-gitlab> gitlab # [ 7.687300] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console vm-test-run-nixbot-gitlab> gitlab # [ 7.701137] Console: switching to colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 7.702565] [drm] Found bochs VGA, ID 0xb0c5. vm-test-run-nixbot-gitlab> gitlab # [ 7.702572] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. vm-test-run-nixbot-gitlab> gitlab # [ 7.705226] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 7.706922] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.711732] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.715628] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD vm-test-run-nixbot-gitlab> gitlab # [ 7.717058] fbcon: bochs-drmdrmfb (fb0) is primary device vm-test-run-nixbot-gitlab> gitlab # [ 7.722116] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 vm-test-run-nixbot-gitlab> gitlab # [ 7.722721] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 7.726368] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 vm-test-run-nixbot-gitlab> gitlab # [ 7.727610] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 7.749612] rtc_cmos 00:05: RTC can wake from S4 vm-test-run-nixbot-gitlab> gitlab # [ 7.750535] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 vm-test-run-nixbot-gitlab> gitlab # [ 7.756558] rtc_cmos 00:05: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 7.756781] parport_pc 00:03: reported by Plug and Play ACPI vm-test-run-nixbot-gitlab> gitlab # [ 7.758053] rtc_cmos 00:05: setting system clock to 2026-07-01T18:01:15 UTC (1782928875) vm-test-run-nixbot-gitlab> gitlab # [ 7.759424] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] vm-test-run-nixbot-gitlab> gitlab # [ 7.762356] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs vm-test-run-nixbot-gitlab> gitlab # [ 7.767560] Floppy drive(s): fd0 is 2.88M AMI BIOS vm-test-run-nixbot-gitlab> gitlab # [ 7.769615] systemd-journald[792]: Time jumped backwards, rotating. vm-test-run-nixbot-gitlab> gitlab # [ 7.784861] FDC 0 is a S82078B vm-test-run-nixbot-gitlab> gitlab # [ 7.618751] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 7.637863] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.641488] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.902706] ppdev: user-space parallel port driver vm-test-run-nixbot-gitlab> gitlab # [ 7.644222] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 7.654573] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.656568] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.660640] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 7.934653] kvm_amd: TSC scaling supported vm-test-run-nixbot-gitlab> gitlab # [ 7.936113] kvm_amd: Nested Virtualization enabled vm-test-run-nixbot-gitlab> gitlab # [ 7.937904] kvm_amd: Nested Paging enabled vm-test-run-nixbot-gitlab> gitlab # [ 7.938837] kvm_amd: LBR virtualization supported vm-test-run-nixbot-gitlab> gitlab # [ 7.939842] kvm_amd: Virtual VMLOAD VMSAVE supported vm-test-run-nixbot-gitlab> gitlab # [ 7.940893] kvm_amd: Virtual GIF supported vm-test-run-nixbot-gitlab> gitlab # [ 7.941828] kvm_amd: Virtual NMI enabled vm-test-run-nixbot-gitlab> gitlab # [ 7.990129] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 7.758648] systemd-vconsole-setup[964]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 7.762635] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.829892] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.854919] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 7.858520] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.862419] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.866903] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 7.871171] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.875611] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 7.881604] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 7.887892] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 7.913230] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.917170] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 7.936123] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 7.940117] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 7.943543] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 7.962221] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 7.971486] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 7.973730] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 7.993817] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 8.043750] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.045711] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.047298] systemd[1]: suid-sgid-wrappers.service: Consumed 139ms CPU time over 178ms wall clock time, 4.3M memory peak, 1.1M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.274823] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.276854] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 8.278523] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 8.280397] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 8.281875] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.283664] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.285126] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.286884] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.298609] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 8.300258] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.302445] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.304198] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.307719] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 8.382208] dbus-broker-launch[1054]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 8.388383] dbus-broker-launch[1054]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 8.393604] dbus-broker-launch[1054]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 8.403242] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 8.407577] dbus-broker-launch[1054]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 8.411152] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 8.414695] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.418279] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.421340] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 8.431245] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 8.437658] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.444779] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 8.456754] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 8.460898] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.28 seconds) vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.28 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 8.466164] logrotate[1059]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 8.467412] logrotate[1059]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 8.469093] logrotate[1059]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 8.471387] logrotate[1059]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # [ 8.472204] logrotate[1059]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # [ 8.472765] logrotate[1059]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab # [ 8.474252] logrotate[1059]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 8.475006] logrotate[1059]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.475920] logrotate[1059]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab # [ 8.477217] logrotate[1059]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.478135] logrotate[1059]: Now: 2026-07-01 18:01 vm-test-run-nixbot-gitlab> gitlab # [ 8.482158] logrotate[1059]: Last rotated at 2026-07-01 18:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.482883] logrotate[1059]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.488096] logrotate[1059]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.489093] logrotate[1059]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 8.489865] logrotate[1059]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.492135] logrotate[1059]: Now: 2026-07-01 18:01 vm-test-run-nixbot-gitlab> gitlab # [ 8.492959] logrotate[1059]: Last rotated at 2026-07-01 18:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.496582] logrotate[1059]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.497387] logrotate[1059]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.498281] logrotate[1059]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1059) vm-test-run-nixbot-gitlab> gitlab # [ 8.500195] logrotate[1059]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.501195] logrotate[1059]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.502098] logrotate[1059]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.502957] logrotate[1059]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1059) vm-test-run-nixbot-gitlab> gitlab # [ 8.504731] logrotate[1059]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.505872] logrotate[1059]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1059) vm-test-run-nixbot-gitlab> gitlab # [ 8.507084] logrotate[1059]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.507884] logrotate[1059]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.509086] logrotate[1059]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.509939] logrotate[1059]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1059) vm-test-run-nixbot-gitlab> gitlab # [ 8.511274] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 8.512497] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 8.514341] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 8.534342] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.536175] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.537961] systemd[1]: nscd.service: Consumed 25ms CPU time over 96ms wall clock time, 7.1M memory peak, 5.7M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.541823] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.560120] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 8.561907] nsncd[1129]: Jul 01 18:01:16.563 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.565591] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.566435] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.567382] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.568302] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.569171] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 8.570110] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 8.570945] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 8.576052] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 8.603390] network-addresses-eth1-start[1152]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 8.611203] network-addresses-eth1-start[1152]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 8.617384] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 8.625420] systemd-logind[1154]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 8.629556] systemd-logind[1154]: Watching system buttons on /dev/input/event2 (Power Button) vm-test-run-nixbot-gitlab> gitlab # [ 8.631879] systemd-logind[1154]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.633321] systemd-logind[1154]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.634346] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 8.636103] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.658335] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.659784] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.666221] dhcpcd[1170]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 8.667251] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 8.667797] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.669730] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.673723] dhcpcd[1219]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 8.675141] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.676772] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.678893] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 8.941831] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 8.942680] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 8.683572] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 8.704118] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 8.710846] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 8.721155] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 8.723633] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 9.040651] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 9.046367] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 9.048769] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 9.051286] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 9.053217] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # [ 9.089466] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 8.832580] redis-server[1244]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 8.833560] redis-server[1244]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 8.834650] dhcpcd[1219]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 8.837219] redis-server[1244]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1244, just started vm-test-run-nixbot-gitlab> gitlab # [ 8.838182] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 8.841632] redis-server[1244]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 8.842639] redis[1244]: monotonic clock: POSIX clock_gettime vm-test-run-nixbot-gitlab> gitlab # [ 8.843834] redis[1244]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 8.844888] redis[1244]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 8.845950] redis[1244]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 8.847212] redis[1244]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 8.848260] nginx-pre-start[1249]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 8.849305] nginx-pre-start[1249]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 8.851779] dhcpcd[1219]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 8.856628] dhcpcd[1219]: DUID 00:01:00:01:31:d8:12:6c:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.857906] dhcpcd[1219]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.859449] dhcpcd[1219]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 8.865817] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 8.867446] postgresql-pre-start[1248]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 8.868090] postgresql-pre-start[1248]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 8.870418] postgresql-pre-start[1248]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 8.871245] postgresql-pre-start[1248]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 8.872126] postgresql-pre-start[1248]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 8.873305] postgresql-pre-start[1248]: Data page checksums are disabled. vm-test-run-nixbot-gitlab> gitlab # [ 8.874394] postgresql-pre-start[1248]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.875359] postgresql-pre-start[1248]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.876265] postgresql-pre-start[1248]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 8.899118] postgresql-pre-start[1248]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 8.932224] postgresql-pre-start[1248]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 9.164146] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.320952] postgresql-pre-start[1248]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 9.324421] postgresql-pre-start[1248]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.469447] gitlab-workhorse[1279]: time="2026-07-01T18:01:17Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 9.470139] gitlab-workhorse[1279]: time="2026-07-01T18:01:17Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 9.472116] gitlab-workhorse[1279]: time="2026-07-01T18:01:17Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 9.515255] postgresql-pre-start[1248]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.549242] dhcpcd[1219]: eth0: soliciting a DHCP lease vm-test-run-nixbot-gitlab> gitlab # [ 9.815521] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 9.568728] dhcpcd[1219]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 9.574236] dhcpcd[1219]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 9.948990] postgresql-pre-start[1248]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.207766] dhcpcd[1219]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 10.209650] dhcpcd[1219]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.211382] dhcpcd[1219]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.212692] dhcpcd[1219]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.214700] dhcpcd[1219]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.298850] postgresql-pre-start[1248]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.299925] postgresql-pre-start[1248]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 10.301463] postgresql-pre-start[1248]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 10.302968] postgresql-pre-start[1248]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 10.304314] postgresql-pre-start[1248]: pg_ctl -D /var/lib/postgresql/17 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 10.413644] postgres[1307]: [1307] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 10.414949] postgres[1307]: [1307] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.416396] postgres[1307]: [1307] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.424207] postgres[1307]: [1307] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 10.437512] postgres[1313]: [1313] LOG: database system was shut down at 2026-07-01 18:01:17 GMT vm-test-run-nixbot-gitlab> gitlab # [ 10.446156] postgres[1307]: [1307] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 10.452851] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 10.458396] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 10.639943] postgresql-setup-start[1324]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.666360] postgresql-setup-start[1338]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.678675] postgresql-setup-start[1340]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.704572] postgresql-setup-start[1345]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.708877] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 10.710459] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 10.713594] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.826664] gitlab-postgresql-start[1353]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.890453] gitlab-postgresql-start[1362]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 10.942894] gitlab-postgresql-start[1364]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 10.943495] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1358]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.945309] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1358]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.959525] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.999729] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.005680] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.008297] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 11.621727] gitaly[1379]: time="2026-07-01T18:01:19.622Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.626510] gitaly[1379]: time="2026-07-01T18:01:19.623Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.628331] gitaly[1379]: time="2026-07-01T18:01:19.623Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.629837] gitaly[1379]: time="2026-07-01T18:01:19.623Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.631379] gitaly[1379]: time="2026-07-01T18:01:19.623Z" level=info msg="Starting Gitaly" pid=1379 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 11.801457] gitaly[1379]: time="2026-07-01T18:01:19.803Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=179 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.802207] gitaly[1379]: time="2026-07-01T18:01:19.803Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.803253] gitaly[1379]: time="2026-07-01T18:01:19.803Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.803755] gitaly[1379]: time="2026-07-01T18:01:19.803Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1379/git-exec-791663695.d/git pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.808823] gitaly[1379]: time="2026-07-01T18:01:19.810Z" level=info msg="finished detecting git version" duration_ms=6 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.809913] gitaly[1379]: time="2026-07-01T18:01:19.810Z" level=info msg="using Git version" pid=1379 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 11.810841] gitaly[1379]: time="2026-07-01T18:01:19.811Z" level=info msg="clearing disk cache object folder" pid=1379 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.811921] gitaly[1379]: time="2026-07-01T18:01:19.811Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3896982608 pid=1379 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.813119] gitaly[1379]: time="2026-07-01T18:01:19.811Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3896982608 pid=1379 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.814112] gitaly[1379]: time="2026-07-01T18:01:19.811Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.814878] gitaly[1379]: time="2026-07-01T18:01:19.811Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.816068] gitaly[1379]: time="2026-07-01T18:01:19.812Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3896982608 clear_duration_ms=0 pid=1379 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.880850] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=info msg="counting repositories" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.881142] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=info msg="starting to count repositories" pid=1379 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.882264] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=info msg="finished tempdir cleaner walk" pid=1379 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 11.883124] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1379 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.883957] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.885135] gitaly[1379]: time="2026-07-01T18:01:19.882Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.886061] gitaly[1379]: time="2026-07-01T18:01:19.883Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1379 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.886834] gitaly[1379]: time="2026-07-01T18:01:19.883Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1379/sock.d/intern pid=1379 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.887977] gitaly[1379]: time="2026-07-01T18:01:19.884Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.889167] gitaly[1379]: time="2026-07-01T18:01:19.884Z" level=info msg="Started Gitaly" duration_ms=261 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 11.889985] gitaly[1379]: time="2026-07-01T18:01:19.884Z" level=info msg="maintenance: daily scheduled" pid=1379 scheduled="2026-07-02 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 12.399327] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1382]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.400950] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1382]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.424369] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1382]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 13.774514] gitaly[1379]: time="2026-07-01T18:01:21.775Z" level=info msg="License database preloaded" duration_ms=2152 pid=1379 vm-test-run-nixbot-gitlab> gitlab # [ 15.255343] dhcpcd[1219]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 15.256729] dhcpcd[1219]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 15.260203] dhcpcd[1219]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 15.357835] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 15.362801] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 42.674941] postgres[1740]: [1740] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 42.676128] postgres[1740]: [1740] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 42.677972] postgres[1740]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 42.680245] postgres[1740]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 42.682445] postgres[1740]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 42.684417] postgres[1740]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 42.686466] postgres[1740]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 42.688168] postgres[1740]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 42.689679] postgres[1740]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 42.692255] postgres[1740]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 42.694151] postgres[1740]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 42.696236] postgres[1740]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 42.698215] postgres[1740]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 42.699588] postgres[1740]: vm-test-run-nixbot-gitlab> gitlab # [ 42.925299] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1382]: W, [2026-07-01T18:01:50.926122 #1382] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 48.187248] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1795]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 48.187590] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1795]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 49.540349] gitlab-workhorse[1279]: time="2026-07-01T18:01:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 59.549697] gitlab-workhorse[1279]: time="2026-07-01T18:02:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 65.588196] postgres[1870]: [1870] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 65.589452] postgres[1870]: [1870] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 69.560242] gitlab-workhorse[1279]: time="2026-07-01T18:02:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 79.561662] gitlab-workhorse[1279]: time="2026-07-01T18:02:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 80.176595] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 80.185474] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 80.192411] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 80.538846] bundle[2073]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 80.547195] bundle[2073]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 80.548710] bundle[2073]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 83.343613] sidekiq-cluster[2072]: {"severity":"INFO","time":"2026-07-01T18:02:31.327Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 83.476993] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.457Z","pid":2073,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 83.478342] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 83.480128] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} vm-test-run-nixbot-gitlab> gitlab # [ 83.481644] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 83.483269] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 83.484967] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 83.486609] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 83.488392] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Master PID: 2073"} vm-test-run-nixbot-gitlab> gitlab # [ 83.490190] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 83.491878] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 83.493485] bundle[2073]: {"timestamp":"2026-07-01T18:02:31.478Z","pid":2073,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 89.563461] gitlab-workhorse[1279]: time="2026-07-01T18:02:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 99.574211] gitlab-workhorse[1279]: time="2026-07-01T18:02:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 106.905283] bundle[2073]: W, [2026-07-01T18:02:54.906320 #2073] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 109.434496] sidekiq-cluster[2103]: 2026-07-01T18:02:57.436Z pid=2103 tid=tz3 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2103, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431480832} vm-test-run-nixbot-gitlab> gitlab # [ 109.583927] gitlab-workhorse[1279]: time="2026-07-01T18:02:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 109.623241] sidekiq-cluster[2103]: W, [2026-07-01T18:02:57.624987 #2103] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 109.702548] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:57.704Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 111.604472] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.605Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 111.604770] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.606Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 111.629700] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.631Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.633581] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.635Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.637572] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.639Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.641759] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.643Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.645577] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.647Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.649698] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.651Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.656835] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.658Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.660533] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.662Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.664227] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.665Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.671157] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.673Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.676140] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.676Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.681573] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.683Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.690135] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.692Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.694869] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.696Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.699421] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.701Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.703715] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.705Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.708326] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.710Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.713949] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.715Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.717496] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.719Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.720691] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.722Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.724656] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.726Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.727950] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.729Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.731182] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.733Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.734238] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.736Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.737650] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.739Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.740985] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.743Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.744699] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.746Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.748500] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.750Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.751919] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.753Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.755964] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.758Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.758815] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.760Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.762107] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.764Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.765596] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.767Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.769394] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.771Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.772899] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.774Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.776714] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.778Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.779970] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.782Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.783605] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.785Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.787611] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.789Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.791317] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.792Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.792653] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.794Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.796288] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.798Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.799358] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.801Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.802660] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.804Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.806170] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.808Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.809404] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.811Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.812336] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.814Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.815416] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.817Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.818729] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.820Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.822489] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.824Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.825830] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.827Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.829320] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.831Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.832857] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.834Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.841748] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.843Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.845048] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.847Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.848351] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.850Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.851608] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.853Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.855061] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.857Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.858715] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.860Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.861986] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.864Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.865143] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.867Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.868330] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.870Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.871270] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.873Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.874872] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.876Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.878518] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.880Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.881707] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.883Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.885528] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.887Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.888403] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.890Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.891823] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.893Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.894956] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.897Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.898098] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.900Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.901167] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.903Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.904211] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.906Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.907307] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.909Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.910447] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.912Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.913500] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.915Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.916855] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.918Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.920260] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.922Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.923343] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.925Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.926367] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.928Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.929515] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.931Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.932603] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.934Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.935906] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.937Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.939861] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.941Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.943167] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.945Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.946543] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.948Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.949963] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.952Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.953732] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.955Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.957161] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.959Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.960291] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.962Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.963443] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.965Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.966563] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.968Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.969607] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.971Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.972957] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.975Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.976317] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.978Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 111.997062] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:02:59.999Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.000541] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.002Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.004177] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.006Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.007536] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.009Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.018934] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.020Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.021718] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.023Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.024713] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.026Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.046801] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.048Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.054154] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.056Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.060472] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.062Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.065780] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.067Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.068986] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.071Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.072070] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.074Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.076344] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.078Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 112.079778] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:00.081Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 119.595415] gitlab-workhorse[1279]: time="2026-07-01T18:03:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 129.611664] gitlab-workhorse[1279]: time="2026-07-01T18:03:17Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 132.321077] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:20.322Z","message":"Booted Rails 7.2.3 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 132.325585] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:20.327Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 132.327236] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:20.327Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 132.328956] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:20.327Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 132.336823] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:20.338Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 133.597426] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:21.598Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 133.605798] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:21.607Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7bdbf974cd20f632098dd8d8","created_at":"2026-07-01T18:02:27.648Z","trace_propagation_headers":{"sentry-trace":"ebf84105285f46d795433be643a44fd9-96349a6f7d964171","baggage":"sentry-trace_id=ebf84105285f46d795433be643a44fd9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e3d9c14d6f46ae3d589cb3509a600402","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T18:02:27.658Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7bdbf974cd20f632098dd8d8: start","job_status":"start","queue_duration_s":53.948666,"scheduling_latency_s":53.948761} vm-test-run-nixbot-gitlab> gitlab # [ 133.910949] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:21.911Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7bdbf974cd20f632098dd8d8","created_at":"2026-07-01T18:02:27.648Z","trace_propagation_headers":{"sentry-trace":"ebf84105285f46d795433be643a44fd9-96349a6f7d964171","baggage":"sentry-trace_id=ebf84105285f46d795433be643a44fd9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"e3d9c14d6f46ae3d589cb3509a600402","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T18:02:27.658Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7bdbf974cd20f632098dd8d8: done: 0.305159 sec","job_status":"done","queue_duration_s":53.948666,"scheduling_latency_s":53.948761,"redis_calls":31,"redis_duration_s":0.017434,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.011225,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003599,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00261,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.111819519,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.305159,"completed_at":"2026-07-01T18:03:21.911Z","load_balancing_strategy":"primary","db_duration_s":0.09517,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 134.121215] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.122Z","pid":2073,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 134.122499] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.122Z","pid":2073,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 134.125304] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.122Z","pid":2073,"message":"! #\u003cThread:0x000072e47a58af80 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 134.128288] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.122Z","pid":2073,"message":"! #\u003cThread:0x000072e47a7ace58 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 134.130900] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.122Z","pid":2073,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 134.134346] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.123Z","pid":2073,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 134.537344] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.538Z","pid":2073,"message":"- Worker 1 (PID: 2607) booted in 0.29s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 134.547152] bundle[2073]: {"timestamp":"2026-07-01T18:03:22.549Z","pid":2073,"message":"- Worker 0 (PID: 2603) booted in 0.32s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 134.550830] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 134.553062] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 134.559533] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 134.584710] setup-gitlab-start[2631]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 136.922144] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:03:24 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2307 vm-test-run-nixbot-gitlab> gitlab # [ 136.930705] setup-gitlab-start[2650]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token vm-test-run-nixbot-gitlab> gitlab # [ 136.940860] setup-gitlab-start[2651]: ++ jq -r .access_token vm-test-run-nixbot-gitlab> gitlab # [ 137.716297] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:03:25 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 772 vm-test-run-nixbot-gitlab> gitlab # [ 137.722572] setup-gitlab-start[2631]: + BEARER=478b47d983d280bb4dc07f47635c976b69750ea7eb8062f280ebac2b42f37242 vm-test-run-nixbot-gitlab> gitlab # [ 137.726421] setup-gitlab-start[2631]: + AUTH='Authorization: Bearer 478b47d983d280bb4dc07f47635c976b69750ea7eb8062f280ebac2b42f37242' vm-test-run-nixbot-gitlab> gitlab # [ 137.729164] setup-gitlab-start[2631]: + curl -fs -X PUT -H 'Authorization: Bearer 478b47d983d280bb4dc07f47635c976b69750ea7eb8062f280ebac2b42f37242' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' vm-test-run-nixbot-gitlab> gitlab # [ 138.492115] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:03:26 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 750 vm-test-run-nixbot-gitlab> gitlab # [ 138.503452] setup-gitlab-start[2631]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear vm-test-run-nixbot-gitlab> gitlab # [ 138.511360] runuser[2670]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 138.885476] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 138.894380] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 138.903344] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 139.463292] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 139.472243] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 139.480831] setup-gitlab-start[2671]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 146.586382] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:34.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b6e66db0070773945d0c06af","created_at":"2026-07-01T18:03:34.548Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.583Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b6e66db0070773945d0c06af: start","job_status":"start","queue_duration_s":0.003434,"scheduling_latency_s":0.003501} vm-test-run-nixbot-gitlab> gitlab # [ 147.072418] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b6e66db0070773945d0c06af","created_at":"2026-07-01T18:03:34.548Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.583Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b6e66db0070773945d0c06af: done: 0.487163 sec","job_status":"done","queue_duration_s":0.003434,"scheduling_latency_s":0.003501,"redis_calls":71,"redis_duration_s":0.161923,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.02411,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.01394,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.12023,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003643,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.196285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001136457,"duration_s":0.487163,"completed_at":"2026-07-01T18:03:35.073Z","load_balancing_strategy":"primary","db_duration_s":0.029811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.140542] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"14fcc97c61f055228b9a44c4","created_at":"2026-07-01T18:03:34.624Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"329fe4885dd84659b223b710a5763bb4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.650Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-14fcc97c61f055228b9a44c4: start","job_status":"start","queue_duration_s":0.467038,"scheduling_latency_s":0.467106} vm-test-run-nixbot-gitlab> gitlab # [ 147.259631] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"14fcc97c61f055228b9a44c4","created_at":"2026-07-01T18:03:34.624Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"329fe4885dd84659b223b710a5763bb4","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.650Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-14fcc97c61f055228b9a44c4: done: 0.143674 sec","job_status":"done","queue_duration_s":0.467038,"scheduling_latency_s":0.467106,"redis_calls":16,"redis_duration_s":0.032264,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.019318,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003485,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001102095,"duration_s":0.143674,"completed_at":"2026-07-01T18:03:35.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.342696] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7fdc202ca5038bedb5e9634a","created_at":"2026-07-01T18:03:34.670Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"876213af9782101e8ad847b2b5bb8bad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T18:03:34.685Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7fdc202ca5038bedb5e9634a: start","job_status":"start","queue_duration_s":0.579943,"scheduling_latency_s":0.579965} vm-test-run-nixbot-gitlab> gitlab # [ 147.363150] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"7fdc202ca5038bedb5e9634a","created_at":"2026-07-01T18:03:34.670Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"876213af9782101e8ad847b2b5bb8bad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T18:03:34.685Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-7fdc202ca5038bedb5e9634a: done: 0.059029 sec","job_status":"done","queue_duration_s":0.579943,"scheduling_latency_s":0.579965,"redis_calls":10,"redis_duration_s":0.012696,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002009,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001388165,"duration_s":0.059029,"completed_at":"2026-07-01T18:03:35.324Z","load_balancing_strategy":"primary","db_duration_s":0.014355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.426327] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e97a0e4fa33c37da9d4f10f7","created_at":"2026-07-01T18:03:34.693Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91cc454c1fa366cf6c6cd6928b4e9e37","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.704Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e97a0e4fa33c37da9d4f10f7: start","job_status":"start","queue_duration_s":0.623712,"scheduling_latency_s":0.623733} vm-test-run-nixbot-gitlab> gitlab # [ 147.448322] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e97a0e4fa33c37da9d4f10f7","created_at":"2026-07-01T18:03:34.693Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91cc454c1fa366cf6c6cd6928b4e9e37","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.704Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e97a0e4fa33c37da9d4f10f7: done: 0.039786 sec","job_status":"done","queue_duration_s":0.623712,"scheduling_latency_s":0.623733,"redis_calls":10,"redis_duration_s":0.012136,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000604,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"duration_s":0.039786,"completed_at":"2026-07-01T18:03:35.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.449428] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e325c3fab9fadef548d207b8","created_at":"2026-07-01T18:03:34.713Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bf3a562cf7079b8a8062a6539893706","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.728Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e325c3fab9fadef548d207b8: start","job_status":"start","queue_duration_s":0.64151,"scheduling_latency_s":0.641531} vm-test-run-nixbot-gitlab> gitlab # [ 147.466485] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e325c3fab9fadef548d207b8","created_at":"2026-07-01T18:03:34.713Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bf3a562cf7079b8a8062a6539893706","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.728Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e325c3fab9fadef548d207b8: done: 0.020593 sec","job_status":"done","queue_duration_s":0.64151,"scheduling_latency_s":0.641531,"redis_calls":12,"redis_duration_s":0.002839,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000478553,"exclusive_lock_hold_duration_s":0.003609956,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.020593,"completed_at":"2026-07-01T18:03:35.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.570159] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2479926d8746d7f20c963ed0","created_at":"2026-07-01T18:03:34.741Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9be8f74488916667a901b98b6d1e6a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.760Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2479926d8746d7f20c963ed0: start","job_status":"start","queue_duration_s":0.631591,"scheduling_latency_s":0.631612} vm-test-run-nixbot-gitlab> gitlab # [ 147.571460] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2479926d8746d7f20c963ed0","created_at":"2026-07-01T18:03:34.741Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9be8f74488916667a901b98b6d1e6a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.760Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2479926d8746d7f20c963ed0: done: 0.030365 sec","job_status":"done","queue_duration_s":0.631591,"scheduling_latency_s":0.631612,"redis_calls":13,"redis_duration_s":0.0032890000000000003,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001748,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009168763,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.030365,"completed_at":"2026-07-01T18:03:35.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004088,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 147.657107] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4ffae6d7a30031524a585897","created_at":"2026-07-01T18:03:34.782Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39995bea605d2ac68070a8e11187c7c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.796Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4ffae6d7a30031524a585897: start","job_status":"start","queue_duration_s":0.627006,"scheduling_latency_s":0.627036} vm-test-run-nixbot-gitlab> gitlab # [ 147.659071] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"4ffae6d7a30031524a585897","created_at":"2026-07-01T18:03:34.782Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39995bea605d2ac68070a8e11187c7c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.796Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-4ffae6d7a30031524a585897: done: 0.198098 sec","job_status":"done","queue_duration_s":0.627006,"scheduling_latency_s":0.627036,"redis_calls":14,"redis_duration_s":0.003384,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.168565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000484419,"exclusive_lock_hold_duration_s":0.177962282,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.198098,"completed_at":"2026-07-01T18:03:35.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.746702] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"62375b1e84b3ffedad40c837","created_at":"2026-07-01T18:03:34.817Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1dc43f3fe54883a3d9f9940fe06ee9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T18:03:34.833Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-62375b1e84b3ffedad40c837: start","job_status":"start","queue_duration_s":0.789654,"scheduling_latency_s":0.789676} vm-test-run-nixbot-gitlab> gitlab # [ 147.748190] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.636Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 147.749162] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"62375b1e84b3ffedad40c837","created_at":"2026-07-01T18:03:34.817Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1dc43f3fe54883a3d9f9940fe06ee9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T18:03:34.833Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-62375b1e84b3ffedad40c837: done: 0.019113 sec","job_status":"done","queue_duration_s":0.789654,"scheduling_latency_s":0.789676,"redis_calls":10,"redis_duration_s":0.002466,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000588,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350044,"duration_s":0.019113,"completed_at":"2026-07-01T18:03:35.642Z","load_balancing_strategy":"primary","db_duration_s":0.000504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.845185] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d142e6f50bfa390bee52ca49","created_at":"2026-07-01T18:03:34.860Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eb9b265ba83b8eb9040ebfa782b7c3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.874Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d142e6f50bfa390bee52ca49: start","job_status":"start","queue_duration_s":0.769346,"scheduling_latency_s":0.769367} vm-test-run-nixbot-gitlab> gitlab # [ 147.846371] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d142e6f50bfa390bee52ca49","created_at":"2026-07-01T18:03:34.860Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2eb9b265ba83b8eb9040ebfa782b7c3c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.874Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d142e6f50bfa390bee52ca49: done: 0.022191 sec","job_status":"done","queue_duration_s":0.769346,"scheduling_latency_s":0.769367,"redis_calls":12,"redis_duration_s":0.002921,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000591,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000421282,"exclusive_lock_hold_duration_s":0.005924776,"duration_s":0.022191,"completed_at":"2026-07-01T18:03:35.666Z","load_balancing_strategy":"primary","db_duration_s":0.003565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 147.938315] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"74fad7b8de85d88935a7c60e","created_at":"2026-07-01T18:03:34.886Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18140647f94c19afd5596c2a83adff3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.897Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-74fad7b8de85d88935a7c60e: start","job_status":"start","queue_duration_s":0.769658,"scheduling_latency_s":0.769677} vm-test-run-nixbot-gitlab> gitlab # [ 147.939417] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"74fad7b8de85d88935a7c60e","created_at":"2026-07-01T18:03:34.886Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18140647f94c19afd5596c2a83adff3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.897Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-74fad7b8de85d88935a7c60e: done: 0.018865 sec","job_status":"done","queue_duration_s":0.769658,"scheduling_latency_s":0.769677,"redis_calls":12,"redis_duration_s":0.002611,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000922,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000322667,"duration_s":0.018865,"completed_at":"2026-07-01T18:03:35.686Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.052349] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8d1d4fd025b8493871a033e1","created_at":"2026-07-01T18:03:34.936Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.951Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8d1d4fd025b8493871a033e1: start","job_status":"start","queue_duration_s":0.736352,"scheduling_latency_s":0.736372} vm-test-run-nixbot-gitlab> gitlab # [ 148.053936] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8d1d4fd025b8493871a033e1","created_at":"2026-07-01T18:03:34.936Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.951Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8d1d4fd025b8493871a033e1: done: 0.027345 sec","job_status":"done","queue_duration_s":0.736352,"scheduling_latency_s":0.736372,"redis_calls":19,"redis_duration_s":0.004013,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000896,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000305,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002149,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000328813,"duration_s":0.027345,"completed_at":"2026-07-01T18:03:35.715Z","load_balancing_strategy":"primary","db_duration_s":0.001561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.088205] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4d5475b9f6cc0beff3dac74a","created_at":"2026-07-01T18:03:34.965Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89b67daee1035b0500b6e674167a8a9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.978Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4d5475b9f6cc0beff3dac74a: start","job_status":"start","queue_duration_s":0.737806,"scheduling_latency_s":0.737826} vm-test-run-nixbot-gitlab> gitlab # [ 148.158119] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.726Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 148.159218] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4d5475b9f6cc0beff3dac74a","created_at":"2026-07-01T18:03:34.965Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89b67daee1035b0500b6e674167a8a9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:34.978Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4d5475b9f6cc0beff3dac74a: done: 0.016293 sec","job_status":"done","queue_duration_s":0.737806,"scheduling_latency_s":0.737826,"redis_calls":10,"redis_duration_s":0.002171,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000364572,"duration_s":0.016293,"completed_at":"2026-07-01T18:03:35.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.241044] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"041a43b05f12abc1bcc81ec5","created_at":"2026-07-01T18:03:34.998Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7499809f9e9ec425dbeb0a06e282e716","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.013Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-041a43b05f12abc1bcc81ec5: start","job_status":"start","queue_duration_s":0.720734,"scheduling_latency_s":0.720802} vm-test-run-nixbot-gitlab> gitlab # [ 148.242460] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"041a43b05f12abc1bcc81ec5","created_at":"2026-07-01T18:03:34.998Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7499809f9e9ec425dbeb0a06e282e716","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.013Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-041a43b05f12abc1bcc81ec5: done: 0.030187 sec","job_status":"done","queue_duration_s":0.720734,"scheduling_latency_s":0.720802,"redis_calls":10,"redis_duration_s":0.0023510000000000002,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381333,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.030187,"completed_at":"2026-07-01T18:03:35.764Z","load_balancing_strategy":"primary","db_duration_s":0.01021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.314270] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"df5761dfc2d397a293acff83","created_at":"2026-07-01T18:03:35.025Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51affea53c698f175d602c8b5e3f5a2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.036Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-df5761dfc2d397a293acff83: start","job_status":"start","queue_duration_s":0.729459,"scheduling_latency_s":0.729481} vm-test-run-nixbot-gitlab> gitlab # [ 148.315314] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.777Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 148.316242] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"df5761dfc2d397a293acff83","created_at":"2026-07-01T18:03:35.025Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51affea53c698f175d602c8b5e3f5a2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.036Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-df5761dfc2d397a293acff83: done: 0.01804 sec","job_status":"done","queue_duration_s":0.729459,"scheduling_latency_s":0.729481,"redis_calls":10,"redis_duration_s":0.002473,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000603,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389715,"duration_s":0.01804,"completed_at":"2026-07-01T18:03:35.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.421490] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2c7b023289358f78a050351","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2c7b023289358f78a050351: start","job_status":"start","queue_duration_s":0.743263,"scheduling_latency_s":0.743323} vm-test-run-nixbot-gitlab> gitlab # [ 148.423107] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2c7b023289358f78a050351","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2c7b023289358f78a050351: done: 0.020729 sec","job_status":"done","queue_duration_s":0.743263,"scheduling_latency_s":0.743323,"redis_calls":16,"redis_duration_s":0.003834,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000424,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002202,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381334,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020729,"completed_at":"2026-07-01T18:03:35.805Z","load_balancing_strategy":"primary","db_duration_s":0.000395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.500780] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08a8e831956c8327940de56d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08a8e831956c8327940de56d: start","job_status":"start","queue_duration_s":0.765307,"scheduling_latency_s":0.765328} vm-test-run-nixbot-gitlab> gitlab # [ 148.502223] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08a8e831956c8327940de56d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08a8e831956c8327940de56d: done: 0.015647 sec","job_status":"done","queue_duration_s":0.765307,"scheduling_latency_s":0.765328,"redis_calls":13,"redis_duration_s":0.002935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010239,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015647,"completed_at":"2026-07-01T18:03:35.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.584370] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daff99d15fdb17ec3c00b6f5","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daff99d15fdb17ec3c00b6f5: start","job_status":"start","queue_duration_s":0.782225,"scheduling_latency_s":0.782245} vm-test-run-nixbot-gitlab> gitlab # [ 148.585958] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daff99d15fdb17ec3c00b6f5","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daff99d15fdb17ec3c00b6f5: done: 0.015233 sec","job_status":"done","queue_duration_s":0.782225,"scheduling_latency_s":0.782245,"redis_calls":13,"redis_duration_s":0.002728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015233,"completed_at":"2026-07-01T18:03:35.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.657784] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bfaa49ec56c5bbf7f7e8c5d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bfaa49ec56c5bbf7f7e8c5d: start","job_status":"start","queue_duration_s":0.798815,"scheduling_latency_s":0.798836} vm-test-run-nixbot-gitlab> gitlab # [ 148.659060] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bfaa49ec56c5bbf7f7e8c5d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bfaa49ec56c5bbf7f7e8c5d: done: 0.015387 sec","job_status":"done","queue_duration_s":0.798815,"scheduling_latency_s":0.798836,"redis_calls":13,"redis_duration_s":0.002985,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015387,"completed_at":"2026-07-01T18:03:35.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.727371] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"506dba1142cf7246d6253d12","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-506dba1142cf7246d6253d12: start","job_status":"start","queue_duration_s":0.815774,"scheduling_latency_s":0.815794} vm-test-run-nixbot-gitlab> gitlab # [ 148.728437] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"506dba1142cf7246d6253d12","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-506dba1142cf7246d6253d12: done: 0.015516 sec","job_status":"done","queue_duration_s":0.815774,"scheduling_latency_s":0.815794,"redis_calls":13,"redis_duration_s":0.0030299999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015516,"completed_at":"2026-07-01T18:03:35.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.799315] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9abb3e46b73bf2cf8a93112c","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9abb3e46b73bf2cf8a93112c: start","job_status":"start","queue_duration_s":0.832545,"scheduling_latency_s":0.832566} vm-test-run-nixbot-gitlab> gitlab # [ 148.800990] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9abb3e46b73bf2cf8a93112c","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9abb3e46b73bf2cf8a93112c: done: 0.014985 sec","job_status":"done","queue_duration_s":0.832545,"scheduling_latency_s":0.832566,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014985,"completed_at":"2026-07-01T18:03:35.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.895324] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e6baf1c4f6c5656068b7279","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e6baf1c4f6c5656068b7279: start","job_status":"start","queue_duration_s":0.849012,"scheduling_latency_s":0.849033} vm-test-run-nixbot-gitlab> gitlab # [ 148.896451] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e6baf1c4f6c5656068b7279","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e6baf1c4f6c5656068b7279: done: 0.015172 sec","job_status":"done","queue_duration_s":0.849012,"scheduling_latency_s":0.849033,"redis_calls":13,"redis_duration_s":0.002674,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015172,"completed_at":"2026-07-01T18:03:35.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.917539] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b350df75e760394791ad83eb","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b350df75e760394791ad83eb: start","job_status":"start","queue_duration_s":0.865602,"scheduling_latency_s":0.865623} vm-test-run-nixbot-gitlab> gitlab # [ 148.995375] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b350df75e760394791ad83eb","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b350df75e760394791ad83eb: done: 0.015072 sec","job_status":"done","queue_duration_s":0.865602,"scheduling_latency_s":0.865623,"redis_calls":13,"redis_duration_s":0.002829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015072,"completed_at":"2026-07-01T18:03:35.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 148.997185] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0eba221d2b4854226aadb9cc","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0eba221d2b4854226aadb9cc: start","job_status":"start","queue_duration_s":0.882061,"scheduling_latency_s":0.882081} vm-test-run-nixbot-gitlab> gitlab # [ 149.079061] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0eba221d2b4854226aadb9cc","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0eba221d2b4854226aadb9cc: done: 0.018201 sec","job_status":"done","queue_duration_s":0.882061,"scheduling_latency_s":0.882081,"redis_calls":13,"redis_duration_s":0.003974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018201,"completed_at":"2026-07-01T18:03:35.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.081071] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf1bc8237b4e31de636b08a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf1bc8237b4e31de636b08a: start","job_status":"start","queue_duration_s":0.901887,"scheduling_latency_s":0.901914} vm-test-run-nixbot-gitlab> gitlab # [ 149.095080] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf1bc8237b4e31de636b08a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf1bc8237b4e31de636b08a: done: 0.016534 sec","job_status":"done","queue_duration_s":0.901887,"scheduling_latency_s":0.901914,"redis_calls":13,"redis_duration_s":0.003237,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016534,"completed_at":"2026-07-01T18:03:35.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.166059] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4068eb283be1efaed71e6b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4068eb283be1efaed71e6b: start","job_status":"start","queue_duration_s":0.91989,"scheduling_latency_s":0.919911} vm-test-run-nixbot-gitlab> gitlab # [ 149.176080] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4068eb283be1efaed71e6b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4068eb283be1efaed71e6b: done: 0.015328 sec","job_status":"done","queue_duration_s":0.91989,"scheduling_latency_s":0.919911,"redis_calls":13,"redis_duration_s":0.003118,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015328,"completed_at":"2026-07-01T18:03:35.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.271966] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"843c24e512c741a8a1a2411a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-843c24e512c741a8a1a2411a: start","job_status":"start","queue_duration_s":0.936666,"scheduling_latency_s":0.936686} vm-test-run-nixbot-gitlab> gitlab # [ 149.273082] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"843c24e512c741a8a1a2411a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-843c24e512c741a8a1a2411a: done: 0.016788 sec","job_status":"done","queue_duration_s":0.936666,"scheduling_latency_s":0.936686,"redis_calls":13,"redis_duration_s":0.002999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016788,"completed_at":"2026-07-01T18:03:35.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.342082] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:35.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d74ca0238b39a992156a5fce","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d74ca0238b39a992156a5fce: start","job_status":"start","queue_duration_s":0.954963,"scheduling_latency_s":0.954989} vm-test-run-nixbot-gitlab> gitlab # [ 149.343242] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d74ca0238b39a992156a5fce","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d74ca0238b39a992156a5fce: done: 0.015773 sec","job_status":"done","queue_duration_s":0.954963,"scheduling_latency_s":0.954989,"redis_calls":13,"redis_duration_s":0.002706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015773,"completed_at":"2026-07-01T18:03:36.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.450427] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16da90536267e405321fb0af","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16da90536267e405321fb0af: start","job_status":"start","queue_duration_s":0.97207,"scheduling_latency_s":0.97209} vm-test-run-nixbot-gitlab> gitlab # [ 149.451787] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16da90536267e405321fb0af","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16da90536267e405321fb0af: done: 0.016215 sec","job_status":"done","queue_duration_s":0.97207,"scheduling_latency_s":0.97209,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016215,"completed_at":"2026-07-01T18:03:36.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.520322] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"740b18efc98888c21fc0ec56","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-740b18efc98888c21fc0ec56: start","job_status":"start","queue_duration_s":0.989717,"scheduling_latency_s":0.989741} vm-test-run-nixbot-gitlab> gitlab # [ 149.521404] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"740b18efc98888c21fc0ec56","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-740b18efc98888c21fc0ec56: done: 0.016226 sec","job_status":"done","queue_duration_s":0.989717,"scheduling_latency_s":0.989741,"redis_calls":13,"redis_duration_s":0.003137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016226,"completed_at":"2026-07-01T18:03:36.047Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.592062] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e55a709fea769efb727f4bc","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e55a709fea769efb727f4bc: start","job_status":"start","queue_duration_s":1.007173,"scheduling_latency_s":1.007194} vm-test-run-nixbot-gitlab> gitlab # [ 149.593448] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e55a709fea769efb727f4bc","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e55a709fea769efb727f4bc: done: 0.015898 sec","job_status":"done","queue_duration_s":1.007173,"scheduling_latency_s":1.007194,"redis_calls":13,"redis_duration_s":0.003402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002344,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015898,"completed_at":"2026-07-01T18:03:36.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.669473] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"601933bb70525fee1c983a61","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-601933bb70525fee1c983a61: start","job_status":"start","queue_duration_s":1.024532,"scheduling_latency_s":1.024552} vm-test-run-nixbot-gitlab> gitlab # [ 149.670748] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"601933bb70525fee1c983a61","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-601933bb70525fee1c983a61: done: 0.01552 sec","job_status":"done","queue_duration_s":1.024532,"scheduling_latency_s":1.024552,"redis_calls":13,"redis_duration_s":0.00283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01552,"completed_at":"2026-07-01T18:03:36.081Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.710233] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"987a3c17dc62f55236548ee4","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-987a3c17dc62f55236548ee4: start","job_status":"start","queue_duration_s":1.041402,"scheduling_latency_s":1.041423} vm-test-run-nixbot-gitlab> gitlab # [ 149.778477] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"987a3c17dc62f55236548ee4","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-987a3c17dc62f55236548ee4: done: 0.015502 sec","job_status":"done","queue_duration_s":1.041402,"scheduling_latency_s":1.041423,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015502,"completed_at":"2026-07-01T18:03:36.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.789191] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfa11e3cc2a99144a9683e6d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfa11e3cc2a99144a9683e6d: start","job_status":"start","queue_duration_s":1.058168,"scheduling_latency_s":1.058189} vm-test-run-nixbot-gitlab> gitlab # [ 149.866385] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfa11e3cc2a99144a9683e6d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfa11e3cc2a99144a9683e6d: done: 0.015285 sec","job_status":"done","queue_duration_s":1.058168,"scheduling_latency_s":1.058189,"redis_calls":13,"redis_duration_s":0.002914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015285,"completed_at":"2026-07-01T18:03:36.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.868764] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753bb174df3769ffc02aa95d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753bb174df3769ffc02aa95d: start","job_status":"start","queue_duration_s":1.075047,"scheduling_latency_s":1.075067} vm-test-run-nixbot-gitlab> gitlab # [ 149.950933] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"753bb174df3769ffc02aa95d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-753bb174df3769ffc02aa95d: done: 0.015493 sec","job_status":"done","queue_duration_s":1.075047,"scheduling_latency_s":1.075067,"redis_calls":13,"redis_duration_s":0.002986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015493,"completed_at":"2026-07-01T18:03:36.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 149.953087] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"befd3e1fa02f3a8166cf6c7c","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-befd3e1fa02f3a8166cf6c7c: start","job_status":"start","queue_duration_s":1.092036,"scheduling_latency_s":1.092056} vm-test-run-nixbot-gitlab> gitlab # [ 150.022128] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"befd3e1fa02f3a8166cf6c7c","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-befd3e1fa02f3a8166cf6c7c: done: 0.017078 sec","job_status":"done","queue_duration_s":1.092036,"scheduling_latency_s":1.092056,"redis_calls":13,"redis_duration_s":0.0032509999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017078,"completed_at":"2026-07-01T18:03:36.150Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.030230] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ef4760babd817c8964e29da","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ef4760babd817c8964e29da: start","job_status":"start","queue_duration_s":1.110527,"scheduling_latency_s":1.110547} vm-test-run-nixbot-gitlab> gitlab # [ 150.114384] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ef4760babd817c8964e29da","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ef4760babd817c8964e29da: done: 0.015715 sec","job_status":"done","queue_duration_s":1.110527,"scheduling_latency_s":1.110547,"redis_calls":13,"redis_duration_s":0.0033020000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015715,"completed_at":"2026-07-01T18:03:36.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.116217] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e415573b51283c0bf43a2903","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e415573b51283c0bf43a2903: start","job_status":"start","queue_duration_s":1.127735,"scheduling_latency_s":1.127757} vm-test-run-nixbot-gitlab> gitlab # [ 150.131179] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e415573b51283c0bf43a2903","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e415573b51283c0bf43a2903: done: 0.01549 sec","job_status":"done","queue_duration_s":1.127735,"scheduling_latency_s":1.127757,"redis_calls":13,"redis_duration_s":0.002857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01549,"completed_at":"2026-07-01T18:03:36.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.205369] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e955839b2095db5e61f049","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e955839b2095db5e61f049: start","job_status":"start","queue_duration_s":1.14468,"scheduling_latency_s":1.144701} vm-test-run-nixbot-gitlab> gitlab # [ 150.274981] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e955839b2095db5e61f049","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e955839b2095db5e61f049: done: 0.015263 sec","job_status":"done","queue_duration_s":1.14468,"scheduling_latency_s":1.144701,"redis_calls":13,"redis_duration_s":0.002908,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015263,"completed_at":"2026-07-01T18:03:36.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.276558] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d285a72542a4a7dc3c4a1484","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d285a72542a4a7dc3c4a1484: start","job_status":"start","queue_duration_s":1.161199,"scheduling_latency_s":1.161221} vm-test-run-nixbot-gitlab> gitlab # [ 150.345941] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d285a72542a4a7dc3c4a1484","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d285a72542a4a7dc3c4a1484: done: 0.015329 sec","job_status":"done","queue_duration_s":1.161199,"scheduling_latency_s":1.161221,"redis_calls":13,"redis_duration_s":0.002826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015329,"completed_at":"2026-07-01T18:03:36.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.352082] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d15d6c7fc0925da729022e7f","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d15d6c7fc0925da729022e7f: start","job_status":"start","queue_duration_s":1.177792,"scheduling_latency_s":1.177812} vm-test-run-nixbot-gitlab> gitlab # [ 150.427791] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d15d6c7fc0925da729022e7f","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d15d6c7fc0925da729022e7f: done: 0.01519 sec","job_status":"done","queue_duration_s":1.177792,"scheduling_latency_s":1.177812,"redis_calls":13,"redis_duration_s":0.002689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01519,"completed_at":"2026-07-01T18:03:36.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.432406] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6df954c5310517dd0fd8fb3","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6df954c5310517dd0fd8fb3: start","job_status":"start","queue_duration_s":1.194228,"scheduling_latency_s":1.194248} vm-test-run-nixbot-gitlab> gitlab # [ 150.505247] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6df954c5310517dd0fd8fb3","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6df954c5310517dd0fd8fb3: done: 0.015729 sec","job_status":"done","queue_duration_s":1.194228,"scheduling_latency_s":1.194248,"redis_calls":13,"redis_duration_s":0.0030659999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015729,"completed_at":"2026-07-01T18:03:36.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.513229] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d17adbc18ffda278aa49c9f0","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d17adbc18ffda278aa49c9f0: start","job_status":"start","queue_duration_s":1.211259,"scheduling_latency_s":1.211323} vm-test-run-nixbot-gitlab> gitlab # [ 150.533230] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d17adbc18ffda278aa49c9f0","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d17adbc18ffda278aa49c9f0: done: 0.015757 sec","job_status":"done","queue_duration_s":1.211259,"scheduling_latency_s":1.211323,"redis_calls":13,"redis_duration_s":0.003034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015757,"completed_at":"2026-07-01T18:03:36.268Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.632261] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30573b1d972592e4744fb461","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30573b1d972592e4744fb461: start","job_status":"start","queue_duration_s":1.228406,"scheduling_latency_s":1.228426} vm-test-run-nixbot-gitlab> gitlab # [ 150.633832] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30573b1d972592e4744fb461","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30573b1d972592e4744fb461: done: 0.017597 sec","job_status":"done","queue_duration_s":1.228406,"scheduling_latency_s":1.228426,"redis_calls":13,"redis_duration_s":0.00318,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017597,"completed_at":"2026-07-01T18:03:36.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.677343] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edffbf7360c35127d72c939e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edffbf7360c35127d72c939e: start","job_status":"start","queue_duration_s":1.247675,"scheduling_latency_s":1.247702} vm-test-run-nixbot-gitlab> gitlab # [ 150.747990] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edffbf7360c35127d72c939e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edffbf7360c35127d72c939e: done: 0.015643 sec","job_status":"done","queue_duration_s":1.247675,"scheduling_latency_s":1.247702,"redis_calls":13,"redis_duration_s":0.0029630000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015643,"completed_at":"2026-07-01T18:03:36.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.757155] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27c1c7ac3230e7d0b435197","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27c1c7ac3230e7d0b435197: start","job_status":"start","queue_duration_s":1.264712,"scheduling_latency_s":1.264733} vm-test-run-nixbot-gitlab> gitlab # [ 150.777233] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f27c1c7ac3230e7d0b435197","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f27c1c7ac3230e7d0b435197: done: 0.015039 sec","job_status":"done","queue_duration_s":1.264712,"scheduling_latency_s":1.264733,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-07-01T18:03:36.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.867050] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e70db09082a7b4ec426b54e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e70db09082a7b4ec426b54e: start","job_status":"start","queue_duration_s":1.281124,"scheduling_latency_s":1.281145} vm-test-run-nixbot-gitlab> gitlab # [ 150.868380] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e70db09082a7b4ec426b54e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e70db09082a7b4ec426b54e: done: 0.015039 sec","job_status":"done","queue_duration_s":1.281124,"scheduling_latency_s":1.281145,"redis_calls":13,"redis_duration_s":0.0027489999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-07-01T18:03:36.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 150.950354] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:36.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d80d6e316eda4eb25c78ae8b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d80d6e316eda4eb25c78ae8b: start","job_status":"start","queue_duration_s":1.297574,"scheduling_latency_s":1.297595} vm-test-run-nixbot-gitlab> gitlab # [ 151.268244] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2751378377 wd_nsec: 2751376950 vm-test-run-nixbot-gitlab> gitlab # [ 151.015892] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d80d6e316eda4eb25c78ae8b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d80d6e316eda4eb25c78ae8b: done: 2.678162 sec","job_status":"done","queue_duration_s":1.297574,"scheduling_latency_s":1.297595,"redis_calls":13,"redis_duration_s":0.002919,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.678162,"completed_at":"2026-07-01T18:03:39.017Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.078706] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d929703cd32e81cf62fc57b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d929703cd32e81cf62fc57b: start","job_status":"start","queue_duration_s":4.03855,"scheduling_latency_s":4.038586} vm-test-run-nixbot-gitlab> gitlab # [ 151.095779] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d929703cd32e81cf62fc57b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d929703cd32e81cf62fc57b: done: 0.017221 sec","job_status":"done","queue_duration_s":4.03855,"scheduling_latency_s":4.038586,"redis_calls":13,"redis_duration_s":0.0028120000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017221,"completed_at":"2026-07-01T18:03:39.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.160795] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b402209fb67febc9d7e39f0","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b402209fb67febc9d7e39f0: start","job_status":"start","queue_duration_s":4.120705,"scheduling_latency_s":4.120739} vm-test-run-nixbot-gitlab> gitlab # [ 151.197374] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b402209fb67febc9d7e39f0","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b402209fb67febc9d7e39f0: done: 0.036324 sec","job_status":"done","queue_duration_s":4.120705,"scheduling_latency_s":4.120739,"redis_calls":13,"redis_duration_s":0.0037359999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036324,"completed_at":"2026-07-01T18:03:39.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.261183] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7328495643f555d68179df6a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7328495643f555d68179df6a: start","job_status":"start","queue_duration_s":4.220688,"scheduling_latency_s":4.220727} vm-test-run-nixbot-gitlab> gitlab # [ 151.323212] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7328495643f555d68179df6a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7328495643f555d68179df6a: done: 0.061588 sec","job_status":"done","queue_duration_s":4.220688,"scheduling_latency_s":4.220727,"redis_calls":13,"redis_duration_s":0.007648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002419,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061588,"completed_at":"2026-07-01T18:03:39.324Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.388167] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"477c9c9359689a7f9325717e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-477c9c9359689a7f9325717e: start","job_status":"start","queue_duration_s":4.347252,"scheduling_latency_s":4.347333} vm-test-run-nixbot-gitlab> gitlab # [ 151.429976] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"477c9c9359689a7f9325717e","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-477c9c9359689a7f9325717e: done: 0.042022 sec","job_status":"done","queue_duration_s":4.347252,"scheduling_latency_s":4.347333,"redis_calls":13,"redis_duration_s":0.00794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042022,"completed_at":"2026-07-01T18:03:39.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.494241] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f205b2f93bac7a19d50f720","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f205b2f93bac7a19d50f720: start","job_status":"start","queue_duration_s":4.453871,"scheduling_latency_s":4.453906} vm-test-run-nixbot-gitlab> gitlab # [ 151.554776] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f205b2f93bac7a19d50f720","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f205b2f93bac7a19d50f720: done: 0.060249 sec","job_status":"done","queue_duration_s":4.453871,"scheduling_latency_s":4.453906,"redis_calls":13,"redis_duration_s":0.007617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.060249,"completed_at":"2026-07-01T18:03:39.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.591122] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe5d1406ee2d91d61ea02a3","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe5d1406ee2d91d61ea02a3: start","job_status":"start","queue_duration_s":4.517225,"scheduling_latency_s":4.517352} vm-test-run-nixbot-gitlab> gitlab # [ 151.592271] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbe5d1406ee2d91d61ea02a3","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbe5d1406ee2d91d61ea02a3: done: 0.021017 sec","job_status":"done","queue_duration_s":4.517225,"scheduling_latency_s":4.517352,"redis_calls":13,"redis_duration_s":0.005367,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00176,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003607,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021017,"completed_at":"2026-07-01T18:03:39.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.669515] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b47b10e01ddce9389a09125","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b47b10e01ddce9389a09125: start","job_status":"start","queue_duration_s":4.540174,"scheduling_latency_s":4.540195} vm-test-run-nixbot-gitlab> gitlab # [ 151.737849] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b47b10e01ddce9389a09125","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b47b10e01ddce9389a09125: done: 0.1559 sec","job_status":"done","queue_duration_s":4.540174,"scheduling_latency_s":4.540195,"redis_calls":13,"redis_duration_s":0.126956,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.125285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.1559,"completed_at":"2026-07-01T18:03:39.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.739834] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f02ef3aa6258738e21da745","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f02ef3aa6258738e21da745: start","job_status":"start","queue_duration_s":4.699407,"scheduling_latency_s":4.699461} vm-test-run-nixbot-gitlab> gitlab # [ 151.778903] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f02ef3aa6258738e21da745","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f02ef3aa6258738e21da745: done: 0.039323 sec","job_status":"done","queue_duration_s":4.699407,"scheduling_latency_s":4.699461,"redis_calls":13,"redis_duration_s":0.007514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002296,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039323,"completed_at":"2026-07-01T18:03:39.780Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.814949] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81b9b54d41ddc1637f1522d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81b9b54d41ddc1637f1522d: start","job_status":"start","queue_duration_s":4.741124,"scheduling_latency_s":4.741165} vm-test-run-nixbot-gitlab> gitlab # [ 151.834792] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c81b9b54d41ddc1637f1522d","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c81b9b54d41ddc1637f1522d: done: 0.018857 sec","job_status":"done","queue_duration_s":4.741124,"scheduling_latency_s":4.741165,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018857,"completed_at":"2026-07-01T18:03:39.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 151.955170] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2efd61673dc888593a397002","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2efd61673dc888593a397002: start","job_status":"start","queue_duration_s":4.761339,"scheduling_latency_s":4.761361} vm-test-run-nixbot-gitlab> gitlab # [ 151.956527] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2efd61673dc888593a397002","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2efd61673dc888593a397002: done: 0.015496 sec","job_status":"done","queue_duration_s":4.761339,"scheduling_latency_s":4.761361,"redis_calls":13,"redis_duration_s":0.003089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015496,"completed_at":"2026-07-01T18:03:39.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.037339] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3a3625fee9eed72f92a730a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3a3625fee9eed72f92a730a: start","job_status":"start","queue_duration_s":4.778255,"scheduling_latency_s":4.778321} vm-test-run-nixbot-gitlab> gitlab # [ 152.038955] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3a3625fee9eed72f92a730a","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3a3625fee9eed72f92a730a: done: 0.015569 sec","job_status":"done","queue_duration_s":4.778255,"scheduling_latency_s":4.778321,"redis_calls":13,"redis_duration_s":0.0031709999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015569,"completed_at":"2026-07-01T18:03:39.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.059781] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5d32335bca902d2d8f76711","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5d32335bca902d2d8f76711: start","job_status":"start","queue_duration_s":4.795122,"scheduling_latency_s":4.795142} vm-test-run-nixbot-gitlab> gitlab # [ 152.079853] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5d32335bca902d2d8f76711","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5d32335bca902d2d8f76711: done: 0.015649 sec","job_status":"done","queue_duration_s":4.795122,"scheduling_latency_s":4.795142,"redis_calls":13,"redis_duration_s":0.003038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-07-01T18:03:39.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.207634] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a86422ac5ff9809849a60b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a86422ac5ff9809849a60b: start","job_status":"start","queue_duration_s":4.812056,"scheduling_latency_s":4.812076} vm-test-run-nixbot-gitlab> gitlab # [ 152.209128] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a86422ac5ff9809849a60b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a86422ac5ff9809849a60b: done: 0.015309 sec","job_status":"done","queue_duration_s":4.812056,"scheduling_latency_s":4.812076,"redis_calls":13,"redis_duration_s":0.003057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015309,"completed_at":"2026-07-01T18:03:39.869Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.223228] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c737b8252a7fbb666a737b2","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c737b8252a7fbb666a737b2: start","job_status":"start","queue_duration_s":4.828796,"scheduling_latency_s":4.828817} vm-test-run-nixbot-gitlab> gitlab # [ 152.243260] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c737b8252a7fbb666a737b2","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c737b8252a7fbb666a737b2: done: 0.015242 sec","job_status":"done","queue_duration_s":4.828796,"scheduling_latency_s":4.828817,"redis_calls":13,"redis_duration_s":0.002954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015242,"completed_at":"2026-07-01T18:03:39.885Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.306183] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bcaa83e8b46ac7ced61d18f","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bcaa83e8b46ac7ced61d18f: start","job_status":"start","queue_duration_s":4.845482,"scheduling_latency_s":4.845504} vm-test-run-nixbot-gitlab> gitlab # [ 152.388339] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bcaa83e8b46ac7ced61d18f","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bcaa83e8b46ac7ced61d18f: done: 0.016284 sec","job_status":"done","queue_duration_s":4.845482,"scheduling_latency_s":4.845504,"redis_calls":13,"redis_duration_s":0.003302,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002384,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016284,"completed_at":"2026-07-01T18:03:39.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.390187] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3be1b0e6520cffe6d43e67fa","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3be1b0e6520cffe6d43e67fa: start","job_status":"start","queue_duration_s":4.863393,"scheduling_latency_s":4.863414} vm-test-run-nixbot-gitlab> gitlab # [ 152.407184] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3be1b0e6520cffe6d43e67fa","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3be1b0e6520cffe6d43e67fa: done: 0.015672 sec","job_status":"done","queue_duration_s":4.863393,"scheduling_latency_s":4.863414,"redis_calls":13,"redis_duration_s":0.003003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015672,"completed_at":"2026-07-01T18:03:39.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.476949] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186d2d5044f9377693c1d42b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186d2d5044f9377693c1d42b: start","job_status":"start","queue_duration_s":4.880507,"scheduling_latency_s":4.880528} vm-test-run-nixbot-gitlab> gitlab # [ 152.488461] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186d2d5044f9377693c1d42b","created_at":"2026-07-01T18:03:34.730Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-1ab04a16594c47e1","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2c8ffbc28b38bdd1f19f420196a1ae32","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.041Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186d2d5044f9377693c1d42b: done: 0.052012 sec","job_status":"done","queue_duration_s":4.880507,"scheduling_latency_s":4.880528,"redis_calls":13,"redis_duration_s":0.003051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.052012,"completed_at":"2026-07-01T18:03:39.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.551264] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:39.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"51c894290e44ce9a4002d880","created_at":"2026-07-01T18:03:35.059Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f2b5ee89de0d74d875ec67b843fe82","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.075Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-51c894290e44ce9a4002d880: start","job_status":"start","queue_duration_s":4.900344,"scheduling_latency_s":4.900368} vm-test-run-nixbot-gitlab> gitlab # [ 152.624948] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"51c894290e44ce9a4002d880","created_at":"2026-07-01T18:03:35.059Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9f2b5ee89de0d74d875ec67b843fe82","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.075Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-51c894290e44ce9a4002d880: done: 0.514889 sec","job_status":"done","queue_duration_s":4.900344,"scheduling_latency_s":4.900368,"redis_calls":574,"redis_duration_s":0.238218,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.236378,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.302442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403962,"duration_s":0.514889,"completed_at":"2026-07-01T18:03:40.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.640277] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bd297fd43835c64ba7e57a3d","created_at":"2026-07-01T18:03:35.124Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74642f04169f6acb8edb672f05bdf888","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.149Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bd297fd43835c64ba7e57a3d: start","job_status":"start","queue_duration_s":5.342527,"scheduling_latency_s":5.342548} vm-test-run-nixbot-gitlab> gitlab # [ 152.712935] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.503Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 152.714178] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bd297fd43835c64ba7e57a3d","created_at":"2026-07-01T18:03:35.124Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74642f04169f6acb8edb672f05bdf888","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.149Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bd297fd43835c64ba7e57a3d: done: 0.016805 sec","job_status":"done","queue_duration_s":5.342527,"scheduling_latency_s":5.342548,"redis_calls":10,"redis_duration_s":0.00229,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"duration_s":0.016805,"completed_at":"2026-07-01T18:03:40.508Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.734685] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"cf0b510efbe222ffc188ac5d","created_at":"2026-07-01T18:03:35.175Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"980a1948ea94dc835f0f454d84c5aa72","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.192Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-cf0b510efbe222ffc188ac5d: start","job_status":"start","queue_duration_s":5.317863,"scheduling_latency_s":5.317883} vm-test-run-nixbot-gitlab> gitlab # [ 152.816774] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"cf0b510efbe222ffc188ac5d","created_at":"2026-07-01T18:03:35.175Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"980a1948ea94dc835f0f454d84c5aa72","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.192Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-cf0b510efbe222ffc188ac5d: done: 0.029382 sec","job_status":"done","queue_duration_s":5.317863,"scheduling_latency_s":5.317883,"redis_calls":12,"redis_duration_s":0.005794,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001909,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001398781,"exclusive_lock_hold_duration_s":0.010231468,"duration_s":0.029382,"completed_at":"2026-07-01T18:03:40.539Z","load_balancing_strategy":"primary","db_duration_s":0.006794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.821106] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"df57d119442865b1df83fd4b","created_at":"2026-07-01T18:03:35.224Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c86275bd4b835f2d4d159969eca5ad12","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.237Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-df57d119442865b1df83fd4b: start","job_status":"start","queue_duration_s":5.303231,"scheduling_latency_s":5.303252} vm-test-run-nixbot-gitlab> gitlab # [ 152.844846] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"df57d119442865b1df83fd4b","created_at":"2026-07-01T18:03:35.224Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c86275bd4b835f2d4d159969eca5ad12","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.237Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-df57d119442865b1df83fd4b: done: 0.029355 sec","job_status":"done","queue_duration_s":5.303231,"scheduling_latency_s":5.303252,"redis_calls":12,"redis_duration_s":0.002837,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000508,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000458717,"exclusive_lock_hold_duration_s":0.008930744999999999,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.029355,"completed_at":"2026-07-01T18:03:40.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 152.925134] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d9e7419cb4ea643be8642cc6","created_at":"2026-07-01T18:03:35.246Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c54ace6bbc9f17c912d92554125d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.258Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d9e7419cb4ea643be8642cc6: start","job_status":"start","queue_duration_s":5.312999,"scheduling_latency_s":5.31302} vm-test-run-nixbot-gitlab> gitlab # [ 152.937991] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d9e7419cb4ea643be8642cc6","created_at":"2026-07-01T18:03:35.246Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c54ace6bbc9f17c912d92554125d5b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.258Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d9e7419cb4ea643be8642cc6: done: 0.016127 sec","job_status":"done","queue_duration_s":5.312999,"scheduling_latency_s":5.31302,"redis_calls":10,"redis_duration_s":0.002457,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001491,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375187,"duration_s":0.016127,"completed_at":"2026-07-01T18:03:40.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.036311] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1f2bbd8b6bb8d4a3ffae0f25","created_at":"2026-07-01T18:03:35.269Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e718b1847c39b0cf2417dd5e7769aa50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T18:03:35.283Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1f2bbd8b6bb8d4a3ffae0f25: start","job_status":"start","queue_duration_s":5.305589,"scheduling_latency_s":5.305608} vm-test-run-nixbot-gitlab> gitlab # [ 153.037868] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.599Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 153.038497] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1f2bbd8b6bb8d4a3ffae0f25","created_at":"2026-07-01T18:03:35.269Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e718b1847c39b0cf2417dd5e7769aa50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T18:03:35.283Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1f2bbd8b6bb8d4a3ffae0f25: done: 0.016104 sec","job_status":"done","queue_duration_s":5.305589,"scheduling_latency_s":5.305608,"redis_calls":10,"redis_duration_s":0.0024479999999999997,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346971,"duration_s":0.016104,"completed_at":"2026-07-01T18:03:40.605Z","load_balancing_strategy":"primary","db_duration_s":0.000319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.114906] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"87644efb3c341de90436d449","created_at":"2026-07-01T18:03:35.296Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66eeb2a834b9fcf66170ccb1cf5c3a82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.318Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-87644efb3c341de90436d449: start","job_status":"start","queue_duration_s":5.287896,"scheduling_latency_s":5.287916} vm-test-run-nixbot-gitlab> gitlab # [ 153.117212] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"87644efb3c341de90436d449","created_at":"2026-07-01T18:03:35.296Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66eeb2a834b9fcf66170ccb1cf5c3a82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.318Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-87644efb3c341de90436d449: done: 0.168224 sec","job_status":"done","queue_duration_s":5.287896,"scheduling_latency_s":5.287916,"redis_calls":12,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000455924,"exclusive_lock_hold_duration_s":0.150753518,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.168224,"completed_at":"2026-07-01T18:03:40.774Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.240930] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"90493645559ece5226d24a83","created_at":"2026-07-01T18:03:35.327Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77ac376cf2b64600dd47a0b3e9fb360b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.336Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-90493645559ece5226d24a83: start","job_status":"start","queue_duration_s":5.440252,"scheduling_latency_s":5.440272} vm-test-run-nixbot-gitlab> gitlab # [ 153.242240] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.786Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 153.242879] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"90493645559ece5226d24a83","created_at":"2026-07-01T18:03:35.327Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77ac376cf2b64600dd47a0b3e9fb360b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.336Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-90493645559ece5226d24a83: done: 0.016506 sec","job_status":"done","queue_duration_s":5.440252,"scheduling_latency_s":5.440272,"redis_calls":10,"redis_duration_s":0.002123,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339988,"duration_s":0.016506,"completed_at":"2026-07-01T18:03:40.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.322920] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.794Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8375a968e935c55f7df0176f","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8375a968e935c55f7df0176f: start","job_status":"start","queue_duration_s":5.084875,"scheduling_latency_s":5.084895} vm-test-run-nixbot-gitlab> gitlab # [ 153.323743] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.807Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 153.324905] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.816Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8375a968e935c55f7df0176f","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8375a968e935c55f7df0176f: done: 0.021991 sec","job_status":"done","queue_duration_s":5.084875,"scheduling_latency_s":5.084895,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001321,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366527,"duration_s":0.021991,"completed_at":"2026-07-01T18:03:40.816Z","load_balancing_strategy":"primary","db_duration_s":0.002874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.427906] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.817Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c980540c79505e886f951c4a","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c980540c79505e886f951c4a: start","job_status":"start","queue_duration_s":5.108162,"scheduling_latency_s":5.108182} vm-test-run-nixbot-gitlab> gitlab # [ 153.429242] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.830Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 153.430271] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.837Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c980540c79505e886f951c4a","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c980540c79505e886f951c4a: done: 0.019606 sec","job_status":"done","queue_duration_s":5.108162,"scheduling_latency_s":5.108182,"redis_calls":10,"redis_duration_s":0.003471,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001659,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019606,"completed_at":"2026-07-01T18:03:40.836Z","load_balancing_strategy":"primary","db_duration_s":0.001646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.450328] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.838Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03b93ca9a81e154a0d90daf6","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03b93ca9a81e154a0d90daf6: start","job_status":"start","queue_duration_s":5.129208,"scheduling_latency_s":5.129228} vm-test-run-nixbot-gitlab> gitlab # [ 153.531351] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.847Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 153.532457] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.854Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03b93ca9a81e154a0d90daf6","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03b93ca9a81e154a0d90daf6: done: 0.015699 sec","job_status":"done","queue_duration_s":5.129208,"scheduling_latency_s":5.129228,"redis_calls":10,"redis_duration_s":0.002672,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001648,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015699,"completed_at":"2026-07-01T18:03:40.854Z","load_balancing_strategy":"primary","db_duration_s":0.001547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 153.534173] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.855Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"67c8c905585664ce8481873f","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-67c8c905585664ce8481873f: start","job_status":"start","queue_duration_s":5.146217,"scheduling_latency_s":5.146237} vm-test-run-nixbot-gitlab> gitlab # [ 153.552182] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.865Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 153.555093] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:03:40.870Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"67c8c905585664ce8481873f","created_at":"2026-07-01T18:03:35.700Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-cb09ecf0190f4a93","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"91651e086b5dce4f963302881255172b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:03:35.709Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-67c8c905585664ce8481873f: done: 0.015392 sec","job_status":"done","queue_duration_s":5.146217,"scheduling_latency_s":5.146237,"redis_calls":10,"redis_duration_s":0.002369,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001387,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015392,"completed_at":"2026-07-01T18:03:40.870Z","load_balancing_strategy":"primary","db_duration_s":0.001528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.711881] setup-gitlab-start[2671]: W, [2026-07-01T18:03:53.712961 #2671] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 175.344510] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"70ac83e4743f2aad4e9a6657","created_at":"2026-07-01T18:04:03.280Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.342Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-70ac83e4743f2aad4e9a6657: start","job_status":"start","queue_duration_s":0.002858,"scheduling_latency_s":0.002925} vm-test-run-nixbot-gitlab> gitlab # [ 175.521383] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"70ac83e4743f2aad4e9a6657","created_at":"2026-07-01T18:04:03.280Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.342Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-70ac83e4743f2aad4e9a6657: done: 0.17812 sec","job_status":"done","queue_duration_s":0.002858,"scheduling_latency_s":0.002925,"redis_calls":60,"redis_duration_s":0.069706,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.001292,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.061787,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006627,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.17812,"completed_at":"2026-07-01T18:04:03.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 175.600714] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"79f52023e6b443e6c6cb0d0d","created_at":"2026-07-01T18:04:03.356Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"086d0ea50abb440ecdf6182e6c5bfb6a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.359Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-79f52023e6b443e6c6cb0d0d: start","job_status":"start","queue_duration_s":0.242752,"scheduling_latency_s":0.242821} vm-test-run-nixbot-gitlab> gitlab # [ 175.800253] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.801Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"79f52023e6b443e6c6cb0d0d","created_at":"2026-07-01T18:04:03.356Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"086d0ea50abb440ecdf6182e6c5bfb6a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.359Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-79f52023e6b443e6c6cb0d0d: done: 0.199763 sec","job_status":"done","queue_duration_s":0.242752,"scheduling_latency_s":0.242821,"redis_calls":7,"redis_duration_s":0.026739000000000002,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.024299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00244,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.199763,"completed_at":"2026-07-01T18:04:03.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.080781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 175.894456] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.895Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8b650dc6e8994c3535206117","created_at":"2026-07-01T18:04:03.380Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18b0928357f35c2ac66ad0bb3aec62b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T18:04:03.387Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8b650dc6e8994c3535206117: start","job_status":"start","queue_duration_s":0.508395,"scheduling_latency_s":0.508432} vm-test-run-nixbot-gitlab> gitlab # [ 175.977233] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8b650dc6e8994c3535206117","created_at":"2026-07-01T18:04:03.380Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18b0928357f35c2ac66ad0bb3aec62b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-01T18:04:03.387Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8b650dc6e8994c3535206117: done: 0.082747 sec","job_status":"done","queue_duration_s":0.508395,"scheduling_latency_s":0.508432,"redis_calls":8,"redis_duration_s":0.017201,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002408,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082747,"completed_at":"2026-07-01T18:04:03.978Z","load_balancing_strategy":"primary","db_duration_s":0.003738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.020411] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:03.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c816c69cba5caafbdf9c014f","created_at":"2026-07-01T18:04:03.406Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac8a147bbf927712c23fe11e99c09cfc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.411Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c816c69cba5caafbdf9c014f: start","job_status":"start","queue_duration_s":0.576614,"scheduling_latency_s":0.57667} vm-test-run-nixbot-gitlab> gitlab # [ 176.075440] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.076Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c816c69cba5caafbdf9c014f","created_at":"2026-07-01T18:04:03.406Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac8a147bbf927712c23fe11e99c09cfc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.411Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c816c69cba5caafbdf9c014f: done: 0.088557 sec","job_status":"done","queue_duration_s":0.576614,"scheduling_latency_s":0.57667,"redis_calls":7,"redis_duration_s":0.00425,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.088557,"completed_at":"2026-07-01T18:04:04.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.171123] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ec189377888beb9409c0d2cf","created_at":"2026-07-01T18:04:03.422Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cfc5634b55df892f0e811fdb885787a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.426Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ec189377888beb9409c0d2cf: start","job_status":"start","queue_duration_s":0.74589,"scheduling_latency_s":0.74593} vm-test-run-nixbot-gitlab> gitlab # [ 176.264420] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ec189377888beb9409c0d2cf","created_at":"2026-07-01T18:04:03.422Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cfc5634b55df892f0e811fdb885787a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.426Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ec189377888beb9409c0d2cf: done: 0.093129 sec","job_status":"done","queue_duration_s":0.74589,"scheduling_latency_s":0.74593,"redis_calls":9,"redis_duration_s":0.01415,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006987,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001822019,"exclusive_lock_hold_duration_s":0.005326654,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.093129,"completed_at":"2026-07-01T18:04:04.265Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.306473] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.274Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"dd49988c316d6e78232355a4","created_at":"2026-07-01T18:04:03.448Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dfc06078ba7db3d5de9b4c076d53e4a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.456Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-dd49988c316d6e78232355a4: start","job_status":"start","queue_duration_s":0.818205,"scheduling_latency_s":0.818258} vm-test-run-nixbot-gitlab> gitlab # [ 176.337587] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"dd49988c316d6e78232355a4","created_at":"2026-07-01T18:04:03.448Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dfc06078ba7db3d5de9b4c076d53e4a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.456Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-dd49988c316d6e78232355a4: done: 0.064442 sec","job_status":"done","queue_duration_s":0.818205,"scheduling_latency_s":0.818258,"redis_calls":10,"redis_duration_s":0.022164,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006219,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.015945,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007758808,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.064442,"completed_at":"2026-07-01T18:04:04.338Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 176.356339] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d30206eb82805e30c516bb8c","created_at":"2026-07-01T18:04:03.465Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"246ee4c2a1dc9bf86bc680c06de83160","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.467Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d30206eb82805e30c516bb8c: start","job_status":"start","queue_duration_s":0.873861,"scheduling_latency_s":0.873902} vm-test-run-nixbot-gitlab> gitlab # [ 176.428149] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d30206eb82805e30c516bb8c","created_at":"2026-07-01T18:04:03.465Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"246ee4c2a1dc9bf86bc680c06de83160","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.467Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d30206eb82805e30c516bb8c: done: 0.069493 sec","job_status":"done","queue_duration_s":0.873861,"scheduling_latency_s":0.873902,"redis_calls":9,"redis_duration_s":0.002522,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001477,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000477714,"exclusive_lock_hold_duration_s":0.025524753,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.069493,"completed_at":"2026-07-01T18:04:04.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.509440] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"9e98e3fc43888954381af315","created_at":"2026-07-01T18:04:03.479Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91d0d7ac561cef7b1152e8ad26c14c51","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T18:04:03.482Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-9e98e3fc43888954381af315: start","job_status":"start","queue_duration_s":0.930576,"scheduling_latency_s":0.930597} vm-test-run-nixbot-gitlab> gitlab # [ 176.510783] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.420Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 176.511977] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"9e98e3fc43888954381af315","created_at":"2026-07-01T18:04:03.479Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91d0d7ac561cef7b1152e8ad26c14c51","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-01T18:04:03.482Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-9e98e3fc43888954381af315: done: 0.012079 sec","job_status":"done","queue_duration_s":0.930576,"scheduling_latency_s":0.930597,"redis_calls":7,"redis_duration_s":0.001554,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012079,"completed_at":"2026-07-01T18:04:04.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.535945] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"01647edbfa0dbc79385e1fb3","created_at":"2026-07-01T18:04:03.493Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a370fb827d2e180e3abfa7bdb7684807","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.501Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-01647edbfa0dbc79385e1fb3: start","job_status":"start","queue_duration_s":0.924303,"scheduling_latency_s":0.924323} vm-test-run-nixbot-gitlab> gitlab # [ 176.559135] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"01647edbfa0dbc79385e1fb3","created_at":"2026-07-01T18:04:03.493Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a370fb827d2e180e3abfa7bdb7684807","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.501Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-01647edbfa0dbc79385e1fb3: done: 0.025016 sec","job_status":"done","queue_duration_s":0.924303,"scheduling_latency_s":0.924323,"redis_calls":10,"redis_duration_s":0.002571,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341663,"duration_s":0.025016,"completed_at":"2026-07-01T18:04:04.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.635693] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae12d318047fe74466a26b5f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae12d318047fe74466a26b5f: start","job_status":"start","queue_duration_s":0.940621,"scheduling_latency_s":0.940642} vm-test-run-nixbot-gitlab> gitlab # [ 176.645306] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae12d318047fe74466a26b5f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae12d318047fe74466a26b5f: done: 0.015855 sec","job_status":"done","queue_duration_s":0.940621,"scheduling_latency_s":0.940642,"redis_calls":13,"redis_duration_s":0.0033629999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001285,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015855,"completed_at":"2026-07-01T18:04:04.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.749898] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce589de41ae248d59a5ba78","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce589de41ae248d59a5ba78: start","job_status":"start","queue_duration_s":0.95777,"scheduling_latency_s":0.957791} vm-test-run-nixbot-gitlab> gitlab # [ 176.751358] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ce589de41ae248d59a5ba78","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ce589de41ae248d59a5ba78: done: 0.015023 sec","job_status":"done","queue_duration_s":0.95777,"scheduling_latency_s":0.957791,"redis_calls":13,"redis_duration_s":0.002824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015023,"completed_at":"2026-07-01T18:04:04.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.789837] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22812f33626e2cf4aa2d8e1","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22812f33626e2cf4aa2d8e1: start","job_status":"start","queue_duration_s":0.97411,"scheduling_latency_s":0.974131} vm-test-run-nixbot-gitlab> gitlab # [ 176.809865] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22812f33626e2cf4aa2d8e1","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22812f33626e2cf4aa2d8e1: done: 0.01471 sec","job_status":"done","queue_duration_s":0.97411,"scheduling_latency_s":0.974131,"redis_calls":13,"redis_duration_s":0.002758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01471,"completed_at":"2026-07-01T18:04:04.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 176.888106] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e649c5ec6cefd12886ee4f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e649c5ec6cefd12886ee4f: start","job_status":"start","queue_duration_s":0.990101,"scheduling_latency_s":0.990122} vm-test-run-nixbot-gitlab> gitlab # [ 176.890889] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e649c5ec6cefd12886ee4f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e649c5ec6cefd12886ee4f: done: 0.014999 sec","job_status":"done","queue_duration_s":0.990101,"scheduling_latency_s":0.990122,"redis_calls":13,"redis_duration_s":0.002948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-07-01T18:04:04.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.009184] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3e983fa1294f65d2967c9dd","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3e983fa1294f65d2967c9dd: start","job_status":"start","queue_duration_s":1.006713,"scheduling_latency_s":1.006735} vm-test-run-nixbot-gitlab> gitlab # [ 177.010857] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3e983fa1294f65d2967c9dd","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3e983fa1294f65d2967c9dd: done: 0.015649 sec","job_status":"done","queue_duration_s":1.006713,"scheduling_latency_s":1.006735,"redis_calls":13,"redis_duration_s":0.003298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015649,"completed_at":"2026-07-01T18:04:04.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.095064] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bffe2f857f3b8b617f10c3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bffe2f857f3b8b617f10c3: start","job_status":"start","queue_duration_s":1.023719,"scheduling_latency_s":1.023739} vm-test-run-nixbot-gitlab> gitlab # [ 177.095893] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bffe2f857f3b8b617f10c3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bffe2f857f3b8b617f10c3: done: 0.014869 sec","job_status":"done","queue_duration_s":1.023719,"scheduling_latency_s":1.023739,"redis_calls":13,"redis_duration_s":0.002931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014869,"completed_at":"2026-07-01T18:04:04.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.116798] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f236e8cc933fb3d5314ea7cc","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f236e8cc933fb3d5314ea7cc: start","job_status":"start","queue_duration_s":1.040026,"scheduling_latency_s":1.040047} vm-test-run-nixbot-gitlab> gitlab # [ 177.199511] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f236e8cc933fb3d5314ea7cc","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f236e8cc933fb3d5314ea7cc: done: 0.015145 sec","job_status":"done","queue_duration_s":1.040026,"scheduling_latency_s":1.040047,"redis_calls":13,"redis_duration_s":0.002948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015145,"completed_at":"2026-07-01T18:04:04.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.201914] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"269cc399d826ee0ca0d6a936","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-269cc399d826ee0ca0d6a936: start","job_status":"start","queue_duration_s":1.056529,"scheduling_latency_s":1.056549} vm-test-run-nixbot-gitlab> gitlab # [ 177.218263] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"269cc399d826ee0ca0d6a936","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-269cc399d826ee0ca0d6a936: done: 0.015096 sec","job_status":"done","queue_duration_s":1.056529,"scheduling_latency_s":1.056549,"redis_calls":13,"redis_duration_s":0.0030090000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015096,"completed_at":"2026-07-01T18:04:04.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.306464] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41e3c63f2c14b4c281a0baf1","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41e3c63f2c14b4c281a0baf1: start","job_status":"start","queue_duration_s":1.073257,"scheduling_latency_s":1.073278} vm-test-run-nixbot-gitlab> gitlab # [ 177.307901] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41e3c63f2c14b4c281a0baf1","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41e3c63f2c14b4c281a0baf1: done: 0.014857 sec","job_status":"done","queue_duration_s":1.073257,"scheduling_latency_s":1.073278,"redis_calls":13,"redis_duration_s":0.002774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001876,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014857,"completed_at":"2026-07-01T18:04:04.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.380720] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e4af310fca81c5ee653aeb3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e4af310fca81c5ee653aeb3: start","job_status":"start","queue_duration_s":1.089538,"scheduling_latency_s":1.089558} vm-test-run-nixbot-gitlab> gitlab # [ 177.382903] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e4af310fca81c5ee653aeb3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e4af310fca81c5ee653aeb3: done: 0.014897 sec","job_status":"done","queue_duration_s":1.089538,"scheduling_latency_s":1.089558,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014897,"completed_at":"2026-07-01T18:04:04.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.459073] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24e6ed686e7e0a0e03388941","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24e6ed686e7e0a0e03388941: start","job_status":"start","queue_duration_s":1.105638,"scheduling_latency_s":1.105658} vm-test-run-nixbot-gitlab> gitlab # [ 177.531225] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24e6ed686e7e0a0e03388941","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24e6ed686e7e0a0e03388941: done: 0.016195 sec","job_status":"done","queue_duration_s":1.105638,"scheduling_latency_s":1.105658,"redis_calls":13,"redis_duration_s":0.002768,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016195,"completed_at":"2026-07-01T18:04:04.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.533131] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f0c708e4ddbbb9dfcffe860","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f0c708e4ddbbb9dfcffe860: start","job_status":"start","queue_duration_s":1.123122,"scheduling_latency_s":1.123142} vm-test-run-nixbot-gitlab> gitlab # [ 177.615435] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f0c708e4ddbbb9dfcffe860","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f0c708e4ddbbb9dfcffe860: done: 0.015047 sec","job_status":"done","queue_duration_s":1.123122,"scheduling_latency_s":1.123142,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015047,"completed_at":"2026-07-01T18:04:04.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.617284] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4babdf3d9371f6621ba42fe3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4babdf3d9371f6621ba42fe3: start","job_status":"start","queue_duration_s":1.139782,"scheduling_latency_s":1.139802} vm-test-run-nixbot-gitlab> gitlab # [ 177.691277] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4babdf3d9371f6621ba42fe3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4babdf3d9371f6621ba42fe3: done: 0.015006 sec","job_status":"done","queue_duration_s":1.139782,"scheduling_latency_s":1.139802,"redis_calls":13,"redis_duration_s":0.002843,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015006,"completed_at":"2026-07-01T18:04:04.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.692820] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d44c1e7167a17e363fc5de31","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d44c1e7167a17e363fc5de31: start","job_status":"start","queue_duration_s":1.156078,"scheduling_latency_s":1.156099} vm-test-run-nixbot-gitlab> gitlab # [ 177.775365] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d44c1e7167a17e363fc5de31","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d44c1e7167a17e363fc5de31: done: 0.015048 sec","job_status":"done","queue_duration_s":1.156078,"scheduling_latency_s":1.156099,"redis_calls":13,"redis_duration_s":0.0029460000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015048,"completed_at":"2026-07-01T18:04:04.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.777209] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1e1f746c54af66198e95ab7","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1e1f746c54af66198e95ab7: start","job_status":"start","queue_duration_s":1.172403,"scheduling_latency_s":1.172423} vm-test-run-nixbot-gitlab> gitlab # [ 177.860390] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1e1f746c54af66198e95ab7","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1e1f746c54af66198e95ab7: done: 0.015026 sec","job_status":"done","queue_duration_s":1.172403,"scheduling_latency_s":1.172423,"redis_calls":13,"redis_duration_s":0.00289,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015026,"completed_at":"2026-07-01T18:04:04.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.862254] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbbaa8256969a02569e80071","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbbaa8256969a02569e80071: start","job_status":"start","queue_duration_s":1.188698,"scheduling_latency_s":1.188719} vm-test-run-nixbot-gitlab> gitlab # [ 177.876330] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbbaa8256969a02569e80071","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbbaa8256969a02569e80071: done: 0.015344 sec","job_status":"done","queue_duration_s":1.188698,"scheduling_latency_s":1.188719,"redis_calls":13,"redis_duration_s":0.0029639999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015344,"completed_at":"2026-07-01T18:04:04.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 177.949069] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"977a9c903b32fb8454163964","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-977a9c903b32fb8454163964: start","job_status":"start","queue_duration_s":1.205292,"scheduling_latency_s":1.205313} vm-test-run-nixbot-gitlab> gitlab # [ 178.017283] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"977a9c903b32fb8454163964","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-977a9c903b32fb8454163964: done: 0.015417 sec","job_status":"done","queue_duration_s":1.205292,"scheduling_latency_s":1.205313,"redis_calls":13,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015417,"completed_at":"2026-07-01T18:04:04.732Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.019181] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2167a44ba015503b12bd8b97","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2167a44ba015503b12bd8b97: start","job_status":"start","queue_duration_s":1.222016,"scheduling_latency_s":1.222036} vm-test-run-nixbot-gitlab> gitlab # [ 178.087421] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2167a44ba015503b12bd8b97","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2167a44ba015503b12bd8b97: done: 0.015035 sec","job_status":"done","queue_duration_s":1.222016,"scheduling_latency_s":1.222036,"redis_calls":13,"redis_duration_s":0.003058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015035,"completed_at":"2026-07-01T18:04:04.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.097074] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a748dec0b16ebc07bb6a12","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a748dec0b16ebc07bb6a12: start","job_status":"start","queue_duration_s":1.238317,"scheduling_latency_s":1.238337} vm-test-run-nixbot-gitlab> gitlab # [ 178.171449] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a748dec0b16ebc07bb6a12","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a748dec0b16ebc07bb6a12: done: 0.01512 sec","job_status":"done","queue_duration_s":1.238317,"scheduling_latency_s":1.238337,"redis_calls":13,"redis_duration_s":0.002967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-07-01T18:04:04.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.240829] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206ef827b68721e2a25392b2","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206ef827b68721e2a25392b2: start","job_status":"start","queue_duration_s":1.254711,"scheduling_latency_s":1.254731} vm-test-run-nixbot-gitlab> gitlab # [ 178.242222] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"206ef827b68721e2a25392b2","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-206ef827b68721e2a25392b2: done: 0.015028 sec","job_status":"done","queue_duration_s":1.254711,"scheduling_latency_s":1.254731,"redis_calls":13,"redis_duration_s":0.0030499999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015028,"completed_at":"2026-07-01T18:04:04.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.325332] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163ee577e244408b86118fdd","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163ee577e244408b86118fdd: start","job_status":"start","queue_duration_s":1.271037,"scheduling_latency_s":1.271057} vm-test-run-nixbot-gitlab> gitlab # [ 178.326972] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163ee577e244408b86118fdd","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163ee577e244408b86118fdd: done: 0.015392 sec","job_status":"done","queue_duration_s":1.271037,"scheduling_latency_s":1.271057,"redis_calls":13,"redis_duration_s":0.003077,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015392,"completed_at":"2026-07-01T18:04:04.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.398520] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"703432935b00bd66b78c9a10","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-703432935b00bd66b78c9a10: start","job_status":"start","queue_duration_s":1.287701,"scheduling_latency_s":1.287721} vm-test-run-nixbot-gitlab> gitlab # [ 178.400114] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"703432935b00bd66b78c9a10","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-703432935b00bd66b78c9a10: done: 0.015426 sec","job_status":"done","queue_duration_s":1.287701,"scheduling_latency_s":1.287721,"redis_calls":13,"redis_duration_s":0.0030700000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015426,"completed_at":"2026-07-01T18:04:04.815Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.422156] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5586856ff110e1f0ca9529f3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5586856ff110e1f0ca9529f3: start","job_status":"start","queue_duration_s":1.30438,"scheduling_latency_s":1.304401} vm-test-run-nixbot-gitlab> gitlab # [ 178.442367] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5586856ff110e1f0ca9529f3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5586856ff110e1f0ca9529f3: done: 0.014953 sec","job_status":"done","queue_duration_s":1.30438,"scheduling_latency_s":1.304401,"redis_calls":13,"redis_duration_s":0.00293,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014953,"completed_at":"2026-07-01T18:04:04.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.512519] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45e13dcc06ea1c2d44adb8b2","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45e13dcc06ea1c2d44adb8b2: start","job_status":"start","queue_duration_s":1.320656,"scheduling_latency_s":1.320685} vm-test-run-nixbot-gitlab> gitlab # [ 178.585366] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"45e13dcc06ea1c2d44adb8b2","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-45e13dcc06ea1c2d44adb8b2: done: 0.01504 sec","job_status":"done","queue_duration_s":1.320656,"scheduling_latency_s":1.320685,"redis_calls":13,"redis_duration_s":0.002993,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01504,"completed_at":"2026-07-01T18:04:04.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.587180] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2a6a248e9246c753980cf31","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2a6a248e9246c753980cf31: start","job_status":"start","queue_duration_s":1.337341,"scheduling_latency_s":1.337362} vm-test-run-nixbot-gitlab> gitlab # [ 178.604177] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2a6a248e9246c753980cf31","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2a6a248e9246c753980cf31: done: 0.015121 sec","job_status":"done","queue_duration_s":1.337341,"scheduling_latency_s":1.337362,"redis_calls":13,"redis_duration_s":0.0029769999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015121,"completed_at":"2026-07-01T18:04:04.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.676054] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81777ba506d5b8ea5173659f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81777ba506d5b8ea5173659f: start","job_status":"start","queue_duration_s":1.353742,"scheduling_latency_s":1.353763} vm-test-run-nixbot-gitlab> gitlab # [ 178.685106] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81777ba506d5b8ea5173659f","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81777ba506d5b8ea5173659f: done: 0.015041 sec","job_status":"done","queue_duration_s":1.353742,"scheduling_latency_s":1.353763,"redis_calls":13,"redis_duration_s":0.003051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015041,"completed_at":"2026-07-01T18:04:04.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.790213] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f8196e0641468bf6869c6d6","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f8196e0641468bf6869c6d6: start","job_status":"start","queue_duration_s":1.370179,"scheduling_latency_s":1.3702} vm-test-run-nixbot-gitlab> gitlab # [ 178.791868] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f8196e0641468bf6869c6d6","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f8196e0641468bf6869c6d6: done: 0.015076 sec","job_status":"done","queue_duration_s":1.370179,"scheduling_latency_s":1.3702,"redis_calls":13,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015076,"completed_at":"2026-07-01T18:04:04.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.885168] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e761f05894b4e8e8b2dd075","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e761f05894b4e8e8b2dd075: start","job_status":"start","queue_duration_s":1.386523,"scheduling_latency_s":1.386543} vm-test-run-nixbot-gitlab> gitlab # [ 178.886483] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e761f05894b4e8e8b2dd075","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e761f05894b4e8e8b2dd075: done: 0.015287 sec","job_status":"done","queue_duration_s":1.386523,"scheduling_latency_s":1.386543,"redis_calls":13,"redis_duration_s":0.003235,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015287,"completed_at":"2026-07-01T18:04:04.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 178.963299] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37dafd74191875c110b1923a","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37dafd74191875c110b1923a: start","job_status":"start","queue_duration_s":1.403794,"scheduling_latency_s":1.403814} vm-test-run-nixbot-gitlab> gitlab # [ 178.964929] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37dafd74191875c110b1923a","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37dafd74191875c110b1923a: done: 0.01518 sec","job_status":"done","queue_duration_s":1.403794,"scheduling_latency_s":1.403814,"redis_calls":13,"redis_duration_s":0.0029620000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01518,"completed_at":"2026-07-01T18:04:04.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.047348] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d070168e4bcf942b7d0b33d3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d070168e4bcf942b7d0b33d3: start","job_status":"start","queue_duration_s":1.420338,"scheduling_latency_s":1.420359} vm-test-run-nixbot-gitlab> gitlab # [ 179.049116] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d070168e4bcf942b7d0b33d3","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d070168e4bcf942b7d0b33d3: done: 0.01524 sec","job_status":"done","queue_duration_s":1.420338,"scheduling_latency_s":1.420359,"redis_calls":13,"redis_duration_s":0.0030139999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01524,"completed_at":"2026-07-01T18:04:04.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.123210] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"438445100bee32d09be81f9d","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-438445100bee32d09be81f9d: start","job_status":"start","queue_duration_s":1.436929,"scheduling_latency_s":1.436992} vm-test-run-nixbot-gitlab> gitlab # [ 179.124397] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"438445100bee32d09be81f9d","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-438445100bee32d09be81f9d: done: 0.014926 sec","job_status":"done","queue_duration_s":1.436929,"scheduling_latency_s":1.436992,"redis_calls":13,"redis_duration_s":0.002846,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014926,"completed_at":"2026-07-01T18:04:04.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.199177] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9edc11e6707282eec8b0a79b","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9edc11e6707282eec8b0a79b: start","job_status":"start","queue_duration_s":1.453206,"scheduling_latency_s":1.453227} vm-test-run-nixbot-gitlab> gitlab # [ 179.200800] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9edc11e6707282eec8b0a79b","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9edc11e6707282eec8b0a79b: done: 0.015235 sec","job_status":"done","queue_duration_s":1.453206,"scheduling_latency_s":1.453227,"redis_calls":13,"redis_duration_s":0.002747,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015235,"completed_at":"2026-07-01T18:04:04.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.292422] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eeb5ce1fb6f03f77390ac44","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eeb5ce1fb6f03f77390ac44: start","job_status":"start","queue_duration_s":1.469702,"scheduling_latency_s":1.469723} vm-test-run-nixbot-gitlab> gitlab # [ 179.294120] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4eeb5ce1fb6f03f77390ac44","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4eeb5ce1fb6f03f77390ac44: done: 0.014873 sec","job_status":"done","queue_duration_s":1.469702,"scheduling_latency_s":1.469723,"redis_calls":13,"redis_duration_s":0.0027689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001859,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014873,"completed_at":"2026-07-01T18:04:04.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.379295] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:04.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e89c11ef26233d6bf91606","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e89c11ef26233d6bf91606: start","job_status":"start","queue_duration_s":1.485777,"scheduling_latency_s":1.485797} vm-test-run-nixbot-gitlab> gitlab # [ 179.381069] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e89c11ef26233d6bf91606","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e89c11ef26233d6bf91606: done: 0.017013 sec","job_status":"done","queue_duration_s":1.485777,"scheduling_latency_s":1.485797,"redis_calls":13,"redis_duration_s":0.0030870000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017013,"completed_at":"2026-07-01T18:04:05.014Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.450603] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f5ebb255e8749e2d70ab202","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f5ebb255e8749e2d70ab202: start","job_status":"start","queue_duration_s":1.504354,"scheduling_latency_s":1.504381} vm-test-run-nixbot-gitlab> gitlab # [ 179.453259] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f5ebb255e8749e2d70ab202","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f5ebb255e8749e2d70ab202: done: 0.015938 sec","job_status":"done","queue_duration_s":1.504354,"scheduling_latency_s":1.504381,"redis_calls":13,"redis_duration_s":0.0031190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015938,"completed_at":"2026-07-01T18:04:05.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.534139] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2d69578c33e310fddbee50e","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2d69578c33e310fddbee50e: start","job_status":"start","queue_duration_s":1.524421,"scheduling_latency_s":1.524442} vm-test-run-nixbot-gitlab> gitlab # [ 179.535442] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2d69578c33e310fddbee50e","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2d69578c33e310fddbee50e: done: 0.015119 sec","job_status":"done","queue_duration_s":1.524421,"scheduling_latency_s":1.524442,"redis_calls":13,"redis_duration_s":0.00275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015119,"completed_at":"2026-07-01T18:04:05.051Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.605178] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b27e34149acec487cb749127","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b27e34149acec487cb749127: start","job_status":"start","queue_duration_s":1.540771,"scheduling_latency_s":1.540792} vm-test-run-nixbot-gitlab> gitlab # [ 179.606783] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b27e34149acec487cb749127","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b27e34149acec487cb749127: done: 0.014669 sec","job_status":"done","queue_duration_s":1.540771,"scheduling_latency_s":1.540792,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014669,"completed_at":"2026-07-01T18:04:05.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.678223] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f96fcea04e79ceb17fd74de0","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f96fcea04e79ceb17fd74de0: start","job_status":"start","queue_duration_s":1.556668,"scheduling_latency_s":1.556697} vm-test-run-nixbot-gitlab> gitlab # [ 179.679379] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f96fcea04e79ceb17fd74de0","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f96fcea04e79ceb17fd74de0: done: 0.014683 sec","job_status":"done","queue_duration_s":1.556668,"scheduling_latency_s":1.556697,"redis_calls":13,"redis_duration_s":0.002761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014683,"completed_at":"2026-07-01T18:04:05.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.748953] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e11e573fd96548f4d3ffef","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e11e573fd96548f4d3ffef: start","job_status":"start","queue_duration_s":1.572835,"scheduling_latency_s":1.572855} vm-test-run-nixbot-gitlab> gitlab # [ 179.750082] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33e11e573fd96548f4d3ffef","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33e11e573fd96548f4d3ffef: done: 0.014809 sec","job_status":"done","queue_duration_s":1.572835,"scheduling_latency_s":1.572855,"redis_calls":13,"redis_duration_s":0.002898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014809,"completed_at":"2026-07-01T18:04:05.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.817927] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d63b3126262ec8b662fa9633","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d63b3126262ec8b662fa9633: start","job_status":"start","queue_duration_s":1.588925,"scheduling_latency_s":1.588987} vm-test-run-nixbot-gitlab> gitlab # [ 179.819075] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d63b3126262ec8b662fa9633","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d63b3126262ec8b662fa9633: done: 0.015333 sec","job_status":"done","queue_duration_s":1.588925,"scheduling_latency_s":1.588987,"redis_calls":13,"redis_duration_s":0.003319,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002437,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015333,"completed_at":"2026-07-01T18:04:05.116Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.900056] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95a7c7f9bef6776c62db6c7e","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95a7c7f9bef6776c62db6c7e: start","job_status":"start","queue_duration_s":1.605577,"scheduling_latency_s":1.605599} vm-test-run-nixbot-gitlab> gitlab # [ 179.901376] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95a7c7f9bef6776c62db6c7e","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95a7c7f9bef6776c62db6c7e: done: 0.014854 sec","job_status":"done","queue_duration_s":1.605577,"scheduling_latency_s":1.605599,"redis_calls":13,"redis_duration_s":0.002958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014854,"completed_at":"2026-07-01T18:04:05.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 179.985153] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f97b5f64e07588bd77132b","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f97b5f64e07588bd77132b: start","job_status":"start","queue_duration_s":1.621776,"scheduling_latency_s":1.621797} vm-test-run-nixbot-gitlab> gitlab # [ 179.986551] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91f97b5f64e07588bd77132b","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91f97b5f64e07588bd77132b: done: 0.017712 sec","job_status":"done","queue_duration_s":1.621776,"scheduling_latency_s":1.621797,"redis_calls":13,"redis_duration_s":0.002855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017712,"completed_at":"2026-07-01T18:04:05.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.075975] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4eab5548a406c4571b2249c","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4eab5548a406c4571b2249c: start","job_status":"start","queue_duration_s":1.640882,"scheduling_latency_s":1.640903} vm-test-run-nixbot-gitlab> gitlab # [ 180.077324] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4eab5548a406c4571b2249c","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4eab5548a406c4571b2249c: done: 0.01492 sec","job_status":"done","queue_duration_s":1.640882,"scheduling_latency_s":1.640903,"redis_calls":13,"redis_duration_s":0.00277,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001853,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01492,"completed_at":"2026-07-01T18:04:05.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.154919] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca6e0699735d385542ea469a","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca6e0699735d385542ea469a: start","job_status":"start","queue_duration_s":1.657121,"scheduling_latency_s":1.657142} vm-test-run-nixbot-gitlab> gitlab # [ 180.156126] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca6e0699735d385542ea469a","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca6e0699735d385542ea469a: done: 0.016172 sec","job_status":"done","queue_duration_s":1.657121,"scheduling_latency_s":1.657142,"redis_calls":13,"redis_duration_s":0.002956,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016172,"completed_at":"2026-07-01T18:04:05.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.227154] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50dc3f94bb0f890af63ccae","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50dc3f94bb0f890af63ccae: start","job_status":"start","queue_duration_s":1.674812,"scheduling_latency_s":1.674834} vm-test-run-nixbot-gitlab> gitlab # [ 180.228449] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50dc3f94bb0f890af63ccae","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50dc3f94bb0f890af63ccae: done: 0.014771 sec","job_status":"done","queue_duration_s":1.674812,"scheduling_latency_s":1.674834,"redis_calls":13,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014771,"completed_at":"2026-07-01T18:04:05.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.296291] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f90784ecdb6e2d6145e567a5","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f90784ecdb6e2d6145e567a5: start","job_status":"start","queue_duration_s":1.690983,"scheduling_latency_s":1.691003} vm-test-run-nixbot-gitlab> gitlab # [ 180.297476] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f90784ecdb6e2d6145e567a5","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f90784ecdb6e2d6145e567a5: done: 0.015427 sec","job_status":"done","queue_duration_s":1.690983,"scheduling_latency_s":1.691003,"redis_calls":13,"redis_duration_s":0.002907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015427,"completed_at":"2026-07-01T18:04:05.218Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.381665] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e15c2cbbb8ba56a553807365","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e15c2cbbb8ba56a553807365: start","job_status":"start","queue_duration_s":1.707648,"scheduling_latency_s":1.70767} vm-test-run-nixbot-gitlab> gitlab # [ 180.382809] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e15c2cbbb8ba56a553807365","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e15c2cbbb8ba56a553807365: done: 0.01515 sec","job_status":"done","queue_duration_s":1.707648,"scheduling_latency_s":1.70767,"redis_calls":13,"redis_duration_s":0.002907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01515,"completed_at":"2026-07-01T18:04:05.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.453220] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c132b1e4593f53b7b7e5ac30","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c132b1e4593f53b7b7e5ac30: start","job_status":"start","queue_duration_s":1.724214,"scheduling_latency_s":1.724236} vm-test-run-nixbot-gitlab> gitlab # [ 180.528095] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c132b1e4593f53b7b7e5ac30","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c132b1e4593f53b7b7e5ac30: done: 0.019248 sec","job_status":"done","queue_duration_s":1.724214,"scheduling_latency_s":1.724236,"redis_calls":13,"redis_duration_s":0.007011,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019248,"completed_at":"2026-07-01T18:04:05.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.530094] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"104bc7c0b1122fd649e24188","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-104bc7c0b1122fd649e24188: start","job_status":"start","queue_duration_s":1.744763,"scheduling_latency_s":1.744784} vm-test-run-nixbot-gitlab> gitlab # [ 180.599598] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"104bc7c0b1122fd649e24188","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-104bc7c0b1122fd649e24188: done: 0.014905 sec","job_status":"done","queue_duration_s":1.744763,"scheduling_latency_s":1.744784,"redis_calls":13,"redis_duration_s":0.0028650000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014905,"completed_at":"2026-07-01T18:04:05.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.671123] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f47edbbfbc96097d7342393","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f47edbbfbc96097d7342393: start","job_status":"start","queue_duration_s":1.761172,"scheduling_latency_s":1.761193} vm-test-run-nixbot-gitlab> gitlab # [ 180.672462] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f47edbbfbc96097d7342393","created_at":"2026-07-01T18:04:03.394Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-692f92df4a7449ac","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"3148f9f1d53fd0a8ae5b0fc4fa0a83c2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.512Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f47edbbfbc96097d7342393: done: 0.015005 sec","job_status":"done","queue_duration_s":1.761172,"scheduling_latency_s":1.761193,"redis_calls":13,"redis_duration_s":0.003016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015005,"completed_at":"2026-07-01T18:04:05.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.741161] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.289Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ad4215abffcf0ae7e9b9777a","created_at":"2026-07-01T18:04:03.511Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"962c5ac530a79f35762a39b980a3bbbd","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.520Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ad4215abffcf0ae7e9b9777a: start","job_status":"start","queue_duration_s":1.76844,"scheduling_latency_s":1.76846} vm-test-run-nixbot-gitlab> gitlab # [ 180.742786] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.305Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"ad4215abffcf0ae7e9b9777a","created_at":"2026-07-01T18:04:03.511Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"962c5ac530a79f35762a39b980a3bbbd","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.520Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-ad4215abffcf0ae7e9b9777a: done: 0.016109 sec","job_status":"done","queue_duration_s":1.76844,"scheduling_latency_s":1.76846,"redis_calls":10,"redis_duration_s":0.002077,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000338591,"duration_s":0.016109,"completed_at":"2026-07-01T18:04:05.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000653,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.817920] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.307Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0929f39bbcf9c7ba7935e038","created_at":"2026-07-01T18:04:03.625Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56da3edec8eaa3f16042270413e7bc22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.630Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0929f39bbcf9c7ba7935e038: start","job_status":"start","queue_duration_s":1.676055,"scheduling_latency_s":1.676076} vm-test-run-nixbot-gitlab> gitlab # [ 180.819269] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.321Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0929f39bbcf9c7ba7935e038","created_at":"2026-07-01T18:04:03.625Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56da3edec8eaa3f16042270413e7bc22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.630Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0929f39bbcf9c7ba7935e038: done: 0.014069 sec","job_status":"done","queue_duration_s":1.676055,"scheduling_latency_s":1.676076,"redis_calls":9,"redis_duration_s":0.0021780000000000002,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000422121,"exclusive_lock_hold_duration_s":0.002232965,"duration_s":0.014069,"completed_at":"2026-07-01T18:04:05.321Z","load_balancing_strategy":"primary","db_duration_s":0.000982,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.889920] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3ca4eac57f1f15370e3c9492","created_at":"2026-07-01T18:04:03.638Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"958b35c392fce76330b65032bba7a813","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.642Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3ca4eac57f1f15370e3c9492: start","job_status":"start","queue_duration_s":1.679949,"scheduling_latency_s":1.679969} vm-test-run-nixbot-gitlab> gitlab # [ 180.891200] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3ca4eac57f1f15370e3c9492","created_at":"2026-07-01T18:04:03.638Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"958b35c392fce76330b65032bba7a813","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.642Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3ca4eac57f1f15370e3c9492: done: 0.011712 sec","job_status":"done","queue_duration_s":1.679949,"scheduling_latency_s":1.679969,"redis_calls":7,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011712,"completed_at":"2026-07-01T18:04:05.334Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 180.960313] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3cb70a8c18b02e7956c99247","created_at":"2026-07-01T18:04:03.684Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.694Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3cb70a8c18b02e7956c99247: start","job_status":"start","queue_duration_s":1.640921,"scheduling_latency_s":1.640942} vm-test-run-nixbot-gitlab> gitlab # [ 180.966847] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"3cb70a8c18b02e7956c99247","created_at":"2026-07-01T18:04:03.684Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.694Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-3cb70a8c18b02e7956c99247: done: 0.019167 sec","job_status":"done","queue_duration_s":1.640921,"scheduling_latency_s":1.640942,"redis_calls":14,"redis_duration_s":0.00304,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000286,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019167,"completed_at":"2026-07-01T18:04:05.354Z","load_balancing_strategy":"primary","db_duration_s":0.000869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.037965] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"157c1600a0f9072a0aa8fb0b","created_at":"2026-07-01T18:04:03.715Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba44eda98c3abd3246ed72b6e5a1aee0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.719Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-157c1600a0f9072a0aa8fb0b: start","job_status":"start","queue_duration_s":1.63602,"scheduling_latency_s":1.636041} vm-test-run-nixbot-gitlab> gitlab # [ 181.113890] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.362Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 181.115249] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"157c1600a0f9072a0aa8fb0b","created_at":"2026-07-01T18:04:03.715Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba44eda98c3abd3246ed72b6e5a1aee0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.719Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-157c1600a0f9072a0aa8fb0b: done: 0.011208 sec","job_status":"done","queue_duration_s":1.63602,"scheduling_latency_s":1.636041,"redis_calls":7,"redis_duration_s":0.00154,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011208,"completed_at":"2026-07-01T18:04:05.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.121522] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ec370466284da4ac7bff4cfb","created_at":"2026-07-01T18:04:03.744Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e88b22e4df4795a97771b4ba3df69a0b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.749Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ec370466284da4ac7bff4cfb: start","job_status":"start","queue_duration_s":1.618933,"scheduling_latency_s":1.618953} vm-test-run-nixbot-gitlab> gitlab # [ 181.144274] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ec370466284da4ac7bff4cfb","created_at":"2026-07-01T18:04:03.744Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e88b22e4df4795a97771b4ba3df69a0b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.749Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ec370466284da4ac7bff4cfb: done: 0.017113 sec","job_status":"done","queue_duration_s":1.618933,"scheduling_latency_s":1.618953,"redis_calls":7,"redis_duration_s":0.0016899999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.017113,"completed_at":"2026-07-01T18:04:05.385Z","load_balancing_strategy":"primary","db_duration_s":0.005364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.215975] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f108a327b71bb122c93fe66f","created_at":"2026-07-01T18:04:03.754Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b36866e848d9e226c3e4f82a893249","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.759Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f108a327b71bb122c93fe66f: start","job_status":"start","queue_duration_s":1.626749,"scheduling_latency_s":1.62677} vm-test-run-nixbot-gitlab> gitlab # [ 181.231355] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.393Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 181.237132] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f108a327b71bb122c93fe66f","created_at":"2026-07-01T18:04:03.754Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5b36866e848d9e226c3e4f82a893249","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.759Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f108a327b71bb122c93fe66f: done: 0.011356 sec","job_status":"done","queue_duration_s":1.626749,"scheduling_latency_s":1.62677,"redis_calls":7,"redis_duration_s":0.001671,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011356,"completed_at":"2026-07-01T18:04:05.398Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.307578] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"af75bf0927b9982703fda672","created_at":"2026-07-01T18:04:03.789Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fcd15e73c67de28e400758f49678a6","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.893Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-af75bf0927b9982703fda672: start","job_status":"start","queue_duration_s":1.505549,"scheduling_latency_s":1.50557} vm-test-run-nixbot-gitlab> gitlab # [ 181.377254] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.415Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"af75bf0927b9982703fda672","created_at":"2026-07-01T18:04:03.789Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4fcd15e73c67de28e400758f49678a6","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.893Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-af75bf0927b9982703fda672: done: 0.016533 sec","job_status":"done","queue_duration_s":1.505549,"scheduling_latency_s":1.50557,"redis_calls":12,"redis_duration_s":0.002844,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.00048,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348089,"duration_s":0.016533,"completed_at":"2026-07-01T18:04:05.415Z","load_balancing_strategy":"primary","db_duration_s":0.000402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.445972] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e0bec7597f25fd279583befa","created_at":"2026-07-01T18:04:03.936Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2d8045d830a57d0c24b8d7d1a437607","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.955Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e0bec7597f25fd279583befa: start","job_status":"start","queue_duration_s":1.461712,"scheduling_latency_s":1.461732} vm-test-run-nixbot-gitlab> gitlab # [ 181.447470] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e0bec7597f25fd279583befa","created_at":"2026-07-01T18:04:03.936Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2d8045d830a57d0c24b8d7d1a437607","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.955Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e0bec7597f25fd279583befa: done: 0.385601 sec","job_status":"done","queue_duration_s":1.461712,"scheduling_latency_s":1.461732,"redis_calls":571,"redis_duration_s":0.12104100000000001,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.119941,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.303223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.385601,"completed_at":"2026-07-01T18:04:05.802Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.526747] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1626a6c1e2a44dd84efcb08c","created_at":"2026-07-01T18:04:03.968Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe5b5273dee0069e798224c9817f5719","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.971Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1626a6c1e2a44dd84efcb08c: start","job_status":"start","queue_duration_s":1.83295,"scheduling_latency_s":1.832976} vm-test-run-nixbot-gitlab> gitlab # [ 181.527967] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.812Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 181.529216] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"1626a6c1e2a44dd84efcb08c","created_at":"2026-07-01T18:04:03.968Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fe5b5273dee0069e798224c9817f5719","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:03.971Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-1626a6c1e2a44dd84efcb08c: done: 0.012781 sec","job_status":"done","queue_duration_s":1.83295,"scheduling_latency_s":1.832976,"redis_calls":7,"redis_duration_s":0.001915,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000722,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012781,"completed_at":"2026-07-01T18:04:05.817Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.631161] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"84ccfa12eacffe3cec609a6f","created_at":"2026-07-01T18:04:03.987Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7ea26478defd2d8879a9b89e3cc1dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.017Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-84ccfa12eacffe3cec609a6f: start","job_status":"start","queue_duration_s":1.801064,"scheduling_latency_s":1.801084} vm-test-run-nixbot-gitlab> gitlab # [ 181.632290] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.840Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"84ccfa12eacffe3cec609a6f","created_at":"2026-07-01T18:04:03.987Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7ea26478defd2d8879a9b89e3cc1dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.017Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-84ccfa12eacffe3cec609a6f: done: 0.021274 sec","job_status":"done","queue_duration_s":1.801064,"scheduling_latency_s":1.801084,"redis_calls":12,"redis_duration_s":0.003264,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001631,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000574654,"exclusive_lock_hold_duration_s":0.003648229,"duration_s":0.021274,"completed_at":"2026-07-01T18:04:05.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.716175] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"041e1701f0f62895800487be","created_at":"2026-07-01T18:04:04.167Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed0970afef9483c5368e269a47303234","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.171Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-041e1701f0f62895800487be: start","job_status":"start","queue_duration_s":1.669859,"scheduling_latency_s":1.66988} vm-test-run-nixbot-gitlab> gitlab # [ 181.717532] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"041e1701f0f62895800487be","created_at":"2026-07-01T18:04:04.167Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed0970afef9483c5368e269a47303234","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.171Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-041e1701f0f62895800487be: done: 0.025536 sec","job_status":"done","queue_duration_s":1.669859,"scheduling_latency_s":1.66988,"redis_calls":9,"redis_duration_s":0.002174,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000464584,"exclusive_lock_hold_duration_s":0.008621208,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.025536,"completed_at":"2026-07-01T18:04:05.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.792302] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"78010f26d4bc0196f28aa245","created_at":"2026-07-01T18:04:04.209Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f64533b889b25bc0cc6e8f8f48a28b22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.214Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-78010f26d4bc0196f28aa245: start","job_status":"start","queue_duration_s":1.65454,"scheduling_latency_s":1.65456} vm-test-run-nixbot-gitlab> gitlab # [ 181.793947] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"78010f26d4bc0196f28aa245","created_at":"2026-07-01T18:04:04.209Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f64533b889b25bc0cc6e8f8f48a28b22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.214Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-78010f26d4bc0196f28aa245: done: 0.012868 sec","job_status":"done","queue_duration_s":1.65454,"scheduling_latency_s":1.65456,"redis_calls":7,"redis_duration_s":0.001777,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012868,"completed_at":"2026-07-01T18:04:05.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.867812] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b7075769cf0effed0b37f691","created_at":"2026-07-01T18:04:04.228Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4787819fbf23410fb5d78872a8f6b91a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T18:04:04.235Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b7075769cf0effed0b37f691: start","job_status":"start","queue_duration_s":1.647218,"scheduling_latency_s":1.647239} vm-test-run-nixbot-gitlab> gitlab # [ 181.869145] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.890Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 181.870087] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b7075769cf0effed0b37f691","created_at":"2026-07-01T18:04:04.228Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4787819fbf23410fb5d78872a8f6b91a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-01T18:04:04.235Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b7075769cf0effed0b37f691: done: 0.011991 sec","job_status":"done","queue_duration_s":1.647218,"scheduling_latency_s":1.647239,"redis_calls":7,"redis_duration_s":0.001648,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011991,"completed_at":"2026-07-01T18:04:05.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 181.946195] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:05.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4329fcd85883f8a7c0f624ad","created_at":"2026-07-01T18:04:04.251Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7467bcb1a47420121cdee5dda2f5735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.256Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4329fcd85883f8a7c0f624ad: start","job_status":"start","queue_duration_s":1.6393,"scheduling_latency_s":1.639321} vm-test-run-nixbot-gitlab> gitlab # [ 181.947835] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4329fcd85883f8a7c0f624ad","created_at":"2026-07-01T18:04:04.251Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7467bcb1a47420121cdee5dda2f5735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.256Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4329fcd85883f8a7c0f624ad: done: 0.157949 sec","job_status":"done","queue_duration_s":1.6393,"scheduling_latency_s":1.639321,"redis_calls":11,"redis_duration_s":0.002707,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000478552,"exclusive_lock_hold_duration_s":0.140822367,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.157949,"completed_at":"2026-07-01T18:04:06.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 182.014057] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4b2d5b0eccb9370579b15735","created_at":"2026-07-01T18:04:04.272Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6936ee8acef1d9748a156c346864fb05","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.279Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4b2d5b0eccb9370579b15735: start","job_status":"start","queue_duration_s":1.776471,"scheduling_latency_s":1.776493} vm-test-run-nixbot-gitlab> gitlab # [ 182.087944] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.063Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 182.089352] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4b2d5b0eccb9370579b15735","created_at":"2026-07-01T18:04:04.272Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-bdd9d927d38e42c8","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6936ee8acef1d9748a156c346864fb05","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:04.279Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4b2d5b0eccb9370579b15735: done: 0.012533 sec","job_status":"done","queue_duration_s":1.776471,"scheduling_latency_s":1.776493,"redis_calls":7,"redis_duration_s":0.001601,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012533,"completed_at":"2026-07-01T18:04:06.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 182.107097] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.070Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9bd011767da3f5beb1b0ab01","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9bd011767da3f5beb1b0ab01: start","job_status":"start","queue_duration_s":0.719722,"scheduling_latency_s":0.719743} vm-test-run-nixbot-gitlab> gitlab # [ 182.175897] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.080Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 182.177068] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.086Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9bd011767da3f5beb1b0ab01","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9bd011767da3f5beb1b0ab01: done: 0.016939 sec","job_status":"done","queue_duration_s":0.719722,"scheduling_latency_s":0.719743,"redis_calls":10,"redis_duration_s":0.002167,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001336,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016939,"completed_at":"2026-07-01T18:04:06.086Z","load_balancing_strategy":"primary","db_duration_s":0.001878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 182.247204] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.088Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af21029fefe63e0c37cd4e18","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af21029fefe63e0c37cd4e18: start","job_status":"start","queue_duration_s":0.737973,"scheduling_latency_s":0.737994} vm-test-run-nixbot-gitlab> gitlab # [ 182.248772] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.098Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 182.249951] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.104Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"af21029fefe63e0c37cd4e18","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-af21029fefe63e0c37cd4e18: done: 0.016373 sec","job_status":"done","queue_duration_s":0.737973,"scheduling_latency_s":0.737994,"redis_calls":10,"redis_duration_s":0.002284,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016373,"completed_at":"2026-07-01T18:04:06.104Z","load_balancing_strategy":"primary","db_duration_s":0.001352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 182.326305] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.106Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8165f8ab70f168dc147b49ed","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8165f8ab70f168dc147b49ed: start","job_status":"start","queue_duration_s":0.755641,"scheduling_latency_s":0.755662} vm-test-run-nixbot-gitlab> gitlab # [ 182.327887] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.116Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 182.329130] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.122Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8165f8ab70f168dc147b49ed","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8165f8ab70f168dc147b49ed: done: 0.016961 sec","job_status":"done","queue_duration_s":0.755641,"scheduling_latency_s":0.755662,"redis_calls":10,"redis_duration_s":0.00232,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016961,"completed_at":"2026-07-01T18:04:06.122Z","load_balancing_strategy":"primary","db_duration_s":0.001324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 182.413369] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.124Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8e00b1b494593942bf47a5e9","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8e00b1b494593942bf47a5e9: start","job_status":"start","queue_duration_s":0.773953,"scheduling_latency_s":0.773973} vm-test-run-nixbot-gitlab> gitlab # [ 182.414407] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.133Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 182.415418] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:06.139Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8e00b1b494593942bf47a5e9","created_at":"2026-07-01T18:04:05.344Z","trace_propagation_headers":{"sentry-trace":"a67ccb4313504f348d559958fb6593dc-2450c494eb7e4dbc","baggage":"sentry-trace_id=a67ccb4313504f348d559958fb6593dc,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74840c8fd9bf9bad8ca0fb011eefe685","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:05.350Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8e00b1b494593942bf47a5e9: done: 0.01536 sec","job_status":"done","queue_duration_s":0.773953,"scheduling_latency_s":0.773973,"redis_calls":10,"redis_duration_s":0.00246,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001534,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01536,"completed_at":"2026-07-01T18:04:06.139Z","load_balancing_strategy":"primary","db_duration_s":0.001331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 187.776551] runuser[2670]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 187.782621] setup-gitlab-start[2891]: ++ jq -r .token vm-test-run-nixbot-gitlab> gitlab # [ 187.786409] setup-gitlab-start[2892]: +++ date -d '+7 days' +%F vm-test-run-nixbot-gitlab> gitlab # [ 187.790948] setup-gitlab-start[2890]: ++ curl -fs -X POST -H 'Authorization: Bearer 478b47d983d280bb4dc07f47635c976b69750ea7eb8062f280ebac2b42f37242' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-08"}' http://gitlab/api/v4/users/1/personal_access_tokens vm-test-run-nixbot-gitlab> gitlab # [ 188.540315] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:16.540Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eb8b196a23393b5d244ccf65","created_at":"2026-07-01T18:04:16.513Z","trace_propagation_headers":{"sentry-trace":"87e1881e899b47f3934889520510e38b-e4234dca067c4d2d","baggage":"sentry-trace_id=87e1881e899b47f3934889520510e38b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMR1KE21QC8NYCH7QPAH4","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4560fea90e1d2cc7efb0a1325f194036f23e3aeec9772c15c3b57311ae8e0e1e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:16.536Z","job_size_bytes":451,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eb8b196a23393b5d244ccf65: start","job_status":"start","queue_duration_s":0.003977,"scheduling_latency_s":0.004045} vm-test-run-nixbot-gitlab> gitlab # [ 188.599402] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:16 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 804 vm-test-run-nixbot-gitlab> gitlab # [ 188.603155] setup-gitlab-start[2631]: + TOKEN=glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m vm-test-run-nixbot-gitlab> gitlab # [ 188.605365] setup-gitlab-start[2631]: + test -n glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m vm-test-run-nixbot-gitlab> gitlab # [ 188.607490] setup-gitlab-start[2631]: + test glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m '!=' null vm-test-run-nixbot-gitlab> gitlab # [ 188.609717] setup-gitlab-start[2631]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 188.611374] setup-gitlab-start[2631]: + echo glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m vm-test-run-nixbot-gitlab> gitlab # [ 188.613409] setup-gitlab-start[2631]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 188.615193] setup-gitlab-start[2631]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects vm-test-run-nixbot-gitlab> gitlab # [ 188.693930] sidekiq-cluster[2103]: {"severity":"WARN","time":"2026-07-01T18:04:16.694Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eb8b196a23393b5d244ccf65","created_at":"2026-07-01T18:04:16.513Z","trace_propagation_headers":{"sentry-trace":"87e1881e899b47f3934889520510e38b-e4234dca067c4d2d","baggage":"sentry-trace_id=87e1881e899b47f3934889520510e38b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMR1KE21QC8NYCH7QPAH4","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4560fea90e1d2cc7efb0a1325f194036f23e3aeec9772c15c3b57311ae8e0e1e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:16.536Z","job_size_bytes":451,"pid":2103,"sidekiq_tid":"lyo7","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eb8b196a23393b5d244ccf65: fail: 0.154029 sec","job_status":"fail","queue_duration_s":0.003977,"scheduling_latency_s":0.004045,"redis_calls":11,"redis_duration_s":0.005825,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001695,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000572978,"duration_s":0.154029,"completed_at":"2026-07-01T18:04:16.694Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.03762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 189.330354] gitaly[1379]: time="2026-07-01T18:04:17.331Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.331" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.331092] gitaly[1379]: time="2026-07-01T18:04:17.332Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.331" grpc.time_ms=0.331 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.416313] gitaly[1379]: time="2026-07-01T18:04:17.418Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.417" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.417509] gitaly[1379]: time="2026-07-01T18:04:17.418Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.417" grpc.time_ms=0.402 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.495152] gitaly[1379]: time="2026-07-01T18:04:17.496Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK diskcache=4fc6e6f2-ae71-431e-92e7-907bd8e4727a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T18:04:27.126" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.457" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.496476] gitaly[1379]: time="2026-07-01T18:04:17.497Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T18:04:27.126" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.457" grpc.time_ms=39.274 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.517188] gitaly[1379]: time="2026-07-01T18:04:17.518Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.187" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.518" pid=1379 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.518556] gitaly[1379]: time="2026-07-01T18:04:17.519Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.187" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.518" grpc.time_ms=0.976 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.539691] gitaly[1379]: time="2026-07-01T18:04:17.541Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:27.196" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.527" grpc.time_ms=14.098 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.575779] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:17.575Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"139efb4752c70ef12ebeacfa","created_at":"2026-07-01T18:04:17.555Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:17.572Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-139efb4752c70ef12ebeacfa: start","job_status":"start","queue_duration_s":0.002592,"scheduling_latency_s":0.002657} vm-test-run-nixbot-gitlab> gitlab # [ 189.602411] gitaly[1379]: time="2026-07-01T18:04:17.604Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.273" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.603" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.603972] gitaly[1379]: time="2026-07-01T18:04:17.604Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.273" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.603" grpc.time_ms=0.257 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.634627] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:17.636Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"139efb4752c70ef12ebeacfa","created_at":"2026-07-01T18:04:17.555Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:17.572Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-139efb4752c70ef12ebeacfa: done: 0.060883 sec","job_status":"done","queue_duration_s":0.002592,"scheduling_latency_s":0.002657,"redis_calls":10,"redis_duration_s":0.005156,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002842,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.031833934,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.060883,"completed_at":"2026-07-01T18:04:17.636Z","load_balancing_strategy":"primary","db_duration_s":0.016011,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 189.712577] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:17.639Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1600cacbd4e4ac156a422206","created_at":"2026-07-01T18:04:17.585Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:17.594Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1600cacbd4e4ac156a422206: start","job_status":"start","queue_duration_s":0.044702,"scheduling_latency_s":0.044763} vm-test-run-nixbot-gitlab> gitlab # [ 189.714913] gitaly[1379]: time="2026-07-01T18:04:17.649Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK diskcache=2fa23310-4732-48fb-ac23-baad749a78b8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T18:04:27.280" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.610" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.716126] gitaly[1379]: time="2026-07-01T18:04:17.650Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=32 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-01T18:04:27.280" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.610" grpc.time_ms=39.008 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.716972] gitaly[1379]: time="2026-07-01T18:04:17.659Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.658" pid=1379 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.786429] gitaly[1379]: time="2026-07-01T18:04:17.659Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.658" grpc.time_ms=0.479 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.787832] gitaly[1379]: time="2026-07-01T18:04:17.670Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:27.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.664" grpc.time_ms=6.343 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.789071] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:17.734Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"1600cacbd4e4ac156a422206","created_at":"2026-07-01T18:04:17.585Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:17.594Z","job_size_bytes":2,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-1600cacbd4e4ac156a422206: done: 0.094471 sec","job_status":"done","queue_duration_s":0.044702,"scheduling_latency_s":0.044763,"redis_calls":14,"redis_duration_s":0.003862,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000907,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001823,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.061627665000000005,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.094471,"completed_at":"2026-07-01T18:04:17.733Z","load_balancing_strategy":"primary","db_duration_s":0.019834,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 189.883470] gitaly[1379]: time="2026-07-01T18:04:17.695Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:27.045" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.695" grpc.time_ms=0.126 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 189.884899] gitaly[1379]: time="2026-07-01T18:04:17.701Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T18:04:27.050" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:17.701" grpc.time_ms=0.163 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 190.094479] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.096Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4cbea1892d93ceab3ae27ee0","created_at":"2026-07-01T18:04:18.083Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.094Z","job_size_bytes":5,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4cbea1892d93ceab3ae27ee0: start","job_status":"start","queue_duration_s":0.001775,"scheduling_latency_s":0.00181} vm-test-run-nixbot-gitlab> gitlab # [ 190.218916] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.220Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4cbea1892d93ceab3ae27ee0","created_at":"2026-07-01T18:04:18.083Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.094Z","job_size_bytes":5,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4cbea1892d93ceab3ae27ee0: done: 0.124176 sec","job_status":"done","queue_duration_s":0.001775,"scheduling_latency_s":0.00181,"redis_calls":17,"redis_duration_s":0.008047,"redis_read_bytes":144,"redis_write_bytes":3399,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002109,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":884,"redis_queues_calls":2,"redis_queues_duration_s":0.000872,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1257,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.004214,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":992,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":16,"db_write_count":1,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000755682,"duration_s":0.124176,"completed_at":"2026-07-01T18:04:18.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032174,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 190.323597] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.324Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"12317552cf5026068d3e8922","created_at":"2026-07-01T18:04:18.195Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.200Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-12317552cf5026068d3e8922: start","job_status":"start","queue_duration_s":0.124139,"scheduling_latency_s":0.124206} vm-test-run-nixbot-gitlab> gitlab # [ 190.364432] gitaly[1379]: time="2026-07-01T18:04:18.364Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMRVJ8YSDXXHPRDDXQVYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:28.030" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:18.351" grpc.time_ms=13.084 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 190.735295] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:18 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 2109 vm-test-run-nixbot-gitlab> gitlab # [ 190.744506] setup-gitlab-start[2631]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 190.748113] setup-gitlab-start[2631]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 190.749115] setup-gitlab-start[2631]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 190.766484] setup-gitlab-start[2935]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 190.771569] setup-gitlab-start[2631]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 190.775363] setup-gitlab-start[2631]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 190.776399] setup-gitlab-start[2631]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 190.779074] setup-gitlab-start[2631]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 190.781414] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.778Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"12317552cf5026068d3e8922","created_at":"2026-07-01T18:04:18.195Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.200Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-12317552cf5026068d3e8922: done: 0.453519 sec","job_status":"done","queue_duration_s":0.124139,"scheduling_latency_s":0.124206,"redis_calls":11,"redis_duration_s":0.0061730000000000005,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00206,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.159,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.359,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.359,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.158,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000573257,"duration_s":0.453519,"completed_at":"2026-07-01T18:04:18.778Z","load_balancing_strategy":"primary","db_duration_s":0.264973,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.785213] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.781Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"3ab1db576eeaaa924e6fd47a","created_at":"2026-07-01T18:04:18.189Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-1cfec49fb53d49f0","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.203Z","job_size_bytes":88,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-3ab1db576eeaaa924e6fd47a: start","job_status":"start","queue_duration_s":0.577589,"scheduling_latency_s":0.577632} vm-test-run-nixbot-gitlab> gitlab # [ 190.869131] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.805Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"3ab1db576eeaaa924e6fd47a","created_at":"2026-07-01T18:04:18.189Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-1cfec49fb53d49f0","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.203Z","job_size_bytes":88,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-3ab1db576eeaaa924e6fd47a: done: 0.023904 sec","job_status":"done","queue_duration_s":0.577589,"scheduling_latency_s":0.577632,"redis_calls":10,"redis_duration_s":0.002183,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000422,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340546,"duration_s":0.023904,"completed_at":"2026-07-01T18:04:18.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 190.952142] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.806Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ff19bdcb7f60aec9d9b707a7","created_at":"2026-07-01T18:04:18.203Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.207Z","job_size_bytes":89,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ff19bdcb7f60aec9d9b707a7: start","job_status":"start","queue_duration_s":0.598876,"scheduling_latency_s":0.598897} vm-test-run-nixbot-gitlab> gitlab # [ 190.953799] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:18.851Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ff19bdcb7f60aec9d9b707a7","created_at":"2026-07-01T18:04:18.203Z","trace_propagation_headers":{"sentry-trace":"63d8e259b96f4d80ae7abc706a23038d-6cb4cb22ffbe438a","baggage":"sentry-trace_id=63d8e259b96f4d80ae7abc706a23038d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMRVJ8YSDXXHPRDDXQVYK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:18.207Z","job_size_bytes":89,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ff19bdcb7f60aec9d9b707a7: done: 0.045043 sec","job_status":"done","queue_duration_s":0.598876,"scheduling_latency_s":0.598897,"redis_calls":11,"redis_duration_s":0.002674,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000391949,"duration_s":0.045043,"completed_at":"2026-07-01T18:04:18.851Z","load_balancing_strategy":"primary","db_duration_s":0.017349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 191.043047] setup-gitlab-start[2631]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 191.044296] gitaly[1379]: time="2026-07-01T18:04:18.986Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV2HJMRE3T75K0807756 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:18.978" grpc.time_ms=8.517 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.045364] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 75 vm-test-run-nixbot-gitlab> gitlab # [ 191.051341] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 91 vm-test-run-nixbot-gitlab> gitlab # [ 191.054559] setup-gitlab-start[2940]: [master (root-commit) efb675e] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 191.055917] setup-gitlab-start[2940]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 191.056497] setup-gitlab-start[2940]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 191.057411] setup-gitlab-start[2631]: + git remote add origin http://oauth2:glpat-prOas3zLgHf5nktV6_36vG86MQp1OjEH.01.0w1tgds0m@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 191.058318] setup-gitlab-start[2631]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 191.167345] gitaly[1379]: time="2026-07-01T18:04:19.169Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:29.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.160" grpc.time_ms=8.24 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.194766] gitaly[1379]: time="2026-07-01T18:04:19.196Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-01T18:04:49.087" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-01T18:04:19.187" grpc.time_ms=8.722 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.209138] gitaly[1379]: time="2026-07-01T18:04:19.211Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F duration_ms=102ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 191.231352] gitaly[1379]: time="2026-07-01T18:04:19.233Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F duration_ms=20ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 191.232476] gitaly[1379]: time="2026-07-01T18:04:19.234Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.106" grpc.time_ms=127.614 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.262850] gitaly[1379]: time="2026-07-01T18:04:19.264Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.264" grpc.time_ms=0.531 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.308626] gitaly[1379]: time="2026-07-01T18:04:19.310Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.310" grpc.time_ms=0.488 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.351584] gitaly[1379]: time="2026-07-01T18:04:19.353Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.353" grpc.time_ms=0.426 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.453387] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:19.454Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 efb675e1dce75529c90e19587e3d54bc59651543 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"091d474512823b27afce30e3","created_at":"2026-07-01T18:04:19.432Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-5923c3d80f704b15","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.450Z","job_size_bytes":185,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-091d474512823b27afce30e3: start","job_status":"start","queue_duration_s":0.004483,"scheduling_latency_s":0.004519} vm-test-run-nixbot-gitlab> gitlab # [ 191.469106] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:19.470Z","message":"changes: \"0000000000000000000000000000000000000000 efb675e1dce75529c90e19587e3d54bc59651543 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 191.494862] gitaly[1379]: time="2026-07-01T18:04:19.496Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:29.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.487" grpc.time_ms=8.873 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.540742] gitaly[1379]: time="2026-07-01T18:04:19.542Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=38 command.spawn_token_fork_ms=36 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:29.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.502" grpc.time_ms=39.734 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.564144] gitaly[1379]: time="2026-07-01T18:04:19.565Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=53 command.spawn_token_fork_ms=25 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:29.030" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:19.510" grpc.time_ms=54.897 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.580899] gitaly[1379]: time="2026-07-01T18:04:19.582Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1558 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-01T18:04:29.068" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:19.568" grpc.time_ms=14.03 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.595709] gitaly[1379]: time="2026-07-01T18:04:19.597Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F duration_ms=204ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 191.599243] gitaly[1379]: time="2026-07-01T18:04:19.598Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.391" grpc.time_ms=206.987 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.639366] gitaly[1379]: time="2026-07-01T18:04:19.640Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F diskcache=ec607021-b98f-4198-8bf1-0f4aab38e3c9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:19.049" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.640295] gitaly[1379]: time="2026-07-01T18:04:19.641Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=192 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13619 command.oublock=32 command.real_time_ms=580 command.spawn_token_fork_ms=0 command.system_time_ms=138 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:19.049" grpc.time_ms=591.128 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.678500] gitlab-workhorse[1279]: time="2026-07-01T18:04:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 191.679349] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:19 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 684 vm-test-run-nixbot-gitlab> gitlab # [ 191.686750] setup-gitlab-start[2944]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 191.687954] setup-gitlab-start[2944]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 191.690710] setup-gitlab-start[2944]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 191.693626] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 191.695504] gitaly[1379]: time="2026-07-01T18:04:19.697Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-01T18:04:29.207" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:19.687" grpc.time_ms=9.503 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.717339] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab # [ 191.739829] gitaly[1379]: time="2026-07-01T18:04:19.741Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.740" grpc.time_ms=1.299 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 191.775824] gitaly[1379]: time="2026-07-01T18:04:19.777Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3941 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:19.777" grpc.time_ms=0.376 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 191.804671] gitaly[1379]: time="2026-07-01T18:04:19.806Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F diskcache=f9265600-29a3-4d40-9b12-d11f78a621ea grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:29.227" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.708" pid=1379 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.823414] gitaly[1379]: time="2026-07-01T18:04:19.825Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4794 command.oublock=8 command.real_time_ms=88 command.spawn_token_fork_ms=1 command.system_time_ms=33 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:29.227" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.708" grpc.time_ms=98.19 pid=1379 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.847184] gitaly[1379]: time="2026-07-01T18:04:19.848Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-01T18:04:29.018" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.848" grpc.time_ms=0.146 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.861838] gitaly[1379]: time="2026-07-01T18:04:19.863Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=939 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-01T18:04:29.023" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:19.853" grpc.time_ms=10.166 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.948793] gitaly[1379]: time="2026-07-01T18:04:19.950Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-01T18:04:49.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:19.945" grpc.time_ms=4.893 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 191.995543] gitaly[1379]: time="2026-07-01T18:04:19.997Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T18:04:29.166" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:19.996" grpc.time_ms=0.372 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.020477] gitaly[1379]: time="2026-07-01T18:04:20.022Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-01T18:04:29.186" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-01T18:04:20.016" grpc.time_ms=5.644 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.044835] gitaly[1379]: time="2026-07-01T18:04:20.046Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:30.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:20.039" grpc.time_ms=7.135 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.086198] gitaly[1379]: time="2026-07-01T18:04:20.087Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:30.047" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:20.077" grpc.time_ms=10.118 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.108128] gitaly[1379]: time="2026-07-01T18:04:20.110Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T18:04:50.098" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.098" grpc.time_ms=11.102 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 193.05 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 192.380886] gitaly[1379]: time="2026-07-01T18:04:20.382Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-01T18:04:30.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.380" grpc.time_ms=1.863 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.471455] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.472Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 efb675e1dce75529c90e19587e3d54bc59651543 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"091d474512823b27afce30e3","created_at":"2026-07-01T18:04:19.432Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-5923c3d80f704b15","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.450Z","job_size_bytes":185,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-091d474512823b27afce30e3: done: 1.017576 sec","job_status":"done","queue_duration_s":0.004483,"scheduling_latency_s":0.004519,"gitaly_calls":13,"gitaly_duration_s":0.502416,"redis_calls":195,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.053003,"redis_read_bytes":1461,"redis_write_bytes":18938,"redis_cache_calls":14,"redis_cache_duration_s":0.00463,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.009064,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001409,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4936,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003667,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.019621,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014612,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.195,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.21,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.389538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000570463,"exclusive_lock_hold_duration_s":0.137418583,"duration_s":1.017576,"completed_at":"2026-07-01T18:04:20.472Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.092729,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 192.558125] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.475Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6130f211056f0bde79e655b1","created_at":"2026-07-01T18:04:19.638Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.645Z","job_size_bytes":46,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6130f211056f0bde79e655b1: start","job_status":"start","queue_duration_s":0.829294,"scheduling_latency_s":0.829335} vm-test-run-nixbot-gitlab> gitlab # [ 192.559592] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.546Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6130f211056f0bde79e655b1","created_at":"2026-07-01T18:04:19.638Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.645Z","job_size_bytes":46,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6130f211056f0bde79e655b1: done: 0.071525 sec","job_status":"done","queue_duration_s":0.829294,"scheduling_latency_s":0.829335,"gitaly_calls":2,"gitaly_duration_s":0.014024,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006832,"redis_read_bytes":68,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00134,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000515,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001729,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00255,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000517664,"duration_s":0.071525,"completed_at":"2026-07-01T18:04:20.546Z","load_balancing_strategy":"primary","db_duration_s":0.009366,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 192.643834] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.547Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1e8e3a72260fe9276db6b434","created_at":"2026-07-01T18:04:19.656Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.660Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1e8e3a72260fe9276db6b434: start","job_status":"start","queue_duration_s":0.887555,"scheduling_latency_s":0.887575} vm-test-run-nixbot-gitlab> gitlab # [ 192.645366] gitaly[1379]: time="2026-07-01T18:04:20.514Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-02T00:04:20.509" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:20.510" grpc.time_ms=4.058 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.646399] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.619Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1e8e3a72260fe9276db6b434","created_at":"2026-07-01T18:04:19.656Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.660Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1e8e3a72260fe9276db6b434: done: 0.071222 sec","job_status":"done","queue_duration_s":0.887555,"scheduling_latency_s":0.887575,"gitaly_calls":1,"gitaly_duration_s":0.027897,"redis_calls":15,"redis_duration_s":0.0034980000000000002,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000608,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05201108,"duration_s":0.071222,"completed_at":"2026-07-01T18:04:20.619Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 192.719456] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.620Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"34c37446d01612162cb6584f","created_at":"2026-07-01T18:04:19.836Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.838Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-34c37446d01612162cb6584f: start","job_status":"start","queue_duration_s":0.782325,"scheduling_latency_s":0.782346} vm-test-run-nixbot-gitlab> gitlab # [ 192.802749] gitaly[1379]: time="2026-07-01T18:04:20.526Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-01T18:04:50.519" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.520" grpc.time_ms=5.874 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.804058] gitaly[1379]: time="2026-07-01T18:04:20.592Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F diskcache=6f5ae380-a31f-4b96-8483-38d384e0f1bd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-02T00:04:20.566" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.566" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.804946] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.657Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"34c37446d01612162cb6584f","created_at":"2026-07-01T18:04:19.836Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:19.838Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-34c37446d01612162cb6584f: done: 0.036902 sec","job_status":"done","queue_duration_s":0.782325,"scheduling_latency_s":0.782346,"gitaly_calls":1,"gitaly_duration_s":0.008093,"redis_calls":16,"redis_duration_s":0.00368,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000277,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001652,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036902,"completed_at":"2026-07-01T18:04:20.657Z","load_balancing_strategy":"primary","db_duration_s":0.002,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 192.879856] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:20.659Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7e5e58768ddcc81cd8105d5c","created_at":"2026-07-01T18:04:20.347Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5411dae047117e80b5bb024a8de3fd346778541cc90b96ea03de6f396bd2ea96","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:20.353Z","job_size_bytes":467,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7e5e58768ddcc81cd8105d5c: start","job_status":"start","queue_duration_s":0.306237,"scheduling_latency_s":0.306265} vm-test-run-nixbot-gitlab> gitlab # [ 192.881242] gitaly[1379]: time="2026-07-01T18:04:20.592Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1288 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-02T00:04:20.566" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.566" grpc.time_ms=25.661 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.952153] gitaly[1379]: time="2026-07-01T18:04:20.648Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-01T18:04:50.641" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.642" grpc.time_ms=6.09 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.953438] gitaly[1379]: time="2026-07-01T18:04:20.694Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:50.693" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.693" grpc.time_ms=0.728 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 192.954316] gitaly[1379]: time="2026-07-01T18:04:20.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T18:04:30.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:20.697" grpc.time_ms=6.013 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.027250] gitaly[1379]: time="2026-07-01T18:04:20.951Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:50.049" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.950" grpc.time_ms=0.988 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.028531] gitaly[1379]: time="2026-07-01T18:04:20.960Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-01T18:04:50.055" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:20.955" grpc.time_ms=5.051 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.040875] gitaly[1379]: time="2026-07-01T18:04:21.042Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:31.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:21.033" grpc.time_ms=9.295 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.362260] nixbot[3045]: {"ts": "2026-07-01T18:04:21.363Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 193.393965] nixbot[3045]: {"ts": "2026-07-01T18:04:21.395Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 193.417450] nixbot[3045]: {"ts": "2026-07-01T18:04:21.419Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 193.423093] nixbot[3045]: {"ts": "2026-07-01T18:04:21.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 193.427056] nixbot[3045]: {"ts": "2026-07-01T18:04:21.428Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 193.432838] nixbot[3045]: {"ts": "2026-07-01T18:04:21.434Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 193.437447] nixbot[3045]: {"ts": "2026-07-01T18:04:21.439Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 193.440618] nixbot[3045]: {"ts": "2026-07-01T18:04:21.442Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 193.445948] nixbot[3045]: {"ts": "2026-07-01T18:04:21.447Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 193.448904] nixbot[3045]: {"ts": "2026-07-01T18:04:21.450Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 193.453591] nixbot[3045]: {"ts": "2026-07-01T18:04:21.455Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 193.459112] nixbot[3045]: {"ts": "2026-07-01T18:04:21.460Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 193.463365] nixbot[3045]: {"ts": "2026-07-01T18:04:21.465Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 193.466509] nixbot[3045]: {"ts": "2026-07-01T18:04:21.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 193.470577] nixbot[3045]: {"ts": "2026-07-01T18:04:21.472Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 193.473400] nixbot[3045]: {"ts": "2026-07-01T18:04:21.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 193.477348] nixbot[3045]: {"ts": "2026-07-01T18:04:21.479Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 193.479318] nixbot[3045]: {"ts": "2026-07-01T18:04:21.481Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 193.482987] nixbot[3045]: {"ts": "2026-07-01T18:04:21.484Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} vm-test-run-nixbot-gitlab> gitlab # [ 193.485178] nixbot[3045]: {"ts": "2026-07-01T18:04:21.487Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 193.602434] gitaly[1379]: time="2026-07-01T18:04:21.604Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T18:04:51.085" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:21.586" grpc.time_ms=17.745 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.616115] gitaly[1379]: time="2026-07-01T18:04:21.618Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMV5KSW3K5V73P18PBZ3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-01T18:04:51.106" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:21.606" grpc.time_ms=11.117 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 193.700944] sidekiq-cluster[2103]: {"severity":"WARN","time":"2026-07-01T18:04:21.702Z","class":"CreatePipelineWorker","correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"efb675e1dce75529c90e19587e3d54bc59651543","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"efb675e1dce75529c90e19587e3d54bc59651543","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 193.723355] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:21.724Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7e5e58768ddcc81cd8105d5c","created_at":"2026-07-01T18:04:20.347Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-cf10805e2d9d4506","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5411dae047117e80b5bb024a8de3fd346778541cc90b96ea03de6f396bd2ea96","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:20.353Z","job_size_bytes":467,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7e5e58768ddcc81cd8105d5c: done: 1.065223 sec","job_status":"done","queue_duration_s":0.306237,"scheduling_latency_s":0.306265,"gitaly_calls":7,"gitaly_duration_s":0.110246,"redis_calls":53,"redis_duration_s":0.120549,"redis_read_bytes":171,"redis_write_bytes":4426,"redis_cache_calls":4,"redis_cache_duration_s":0.001787,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.099316,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003921,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.734016,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000593651,"duration_s":1.065223,"completed_at":"2026-07-01T18:04:21.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08539,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 193.761630] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:21.727Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"89b3f71159e4ae8daff28b7e","created_at":"2026-07-01T18:04:20.533Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-258c6c04219947d4","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:20.537Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-89b3f71159e4ae8daff28b7e: start","job_status":"start","queue_duration_s":1.189508,"scheduling_latency_s":1.189549} vm-test-run-nixbot-gitlab> gitlab # [ 193.799416] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:21.801Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"89b3f71159e4ae8daff28b7e","created_at":"2026-07-01T18:04:20.533Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-258c6c04219947d4","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:20.537Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-89b3f71159e4ae8daff28b7e: done: 0.073997 sec","job_status":"done","queue_duration_s":1.189508,"scheduling_latency_s":1.189549,"redis_calls":32,"redis_duration_s":0.020373,"redis_read_bytes":161,"redis_write_bytes":5976,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.015137,"redis_feature_flag_read_bytes":144,"redis_feature_flag_write_bytes":2362,"redis_queues_calls":3,"redis_queues_duration_s":0.000813,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003361,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":13,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":13,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007610464000000001,"duration_s":0.073997,"completed_at":"2026-07-01T18:04:21.800Z","load_balancing_strategy":"primary","db_duration_s":0.012103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 193.830452] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:21.807Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"835a1ce8c226dacc205efbb6","created_at":"2026-07-01T18:04:21.788Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-d374e6e8456349e1","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:21.793Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-835a1ce8c226dacc205efbb6: start","job_status":"start","queue_duration_s":0.014459,"scheduling_latency_s":0.014481} vm-test-run-nixbot-gitlab> gitlab # [ 193.953449] nixbot[3045]: INFO: Started server process [3045] vm-test-run-nixbot-gitlab> gitlab # [ 193.954697] nixbot[3045]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) vm-test-run-nixbot-gitlab> gitlab # [ 193.960259] nixbot[3045]: INFO: ::ffff:127.0.0.1:50164 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 193.965507] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 193.967063] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 193.971707] systemd[1]: Startup finished in 1.574s (kernel) + 4.295s (initrd) + 3min 8.100s (userspace) = 3min 13.970s. vm-test-run-nixbot-gitlab> gitlab # [ 193.973413] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:21.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"835a1ce8c226dacc205efbb6","created_at":"2026-07-01T18:04:21.788Z","trace_propagation_headers":{"sentry-trace":"cf764056b436403081ca3d2af5d263fb-d374e6e8456349e1","baggage":"sentry-trace_id=cf764056b436403081ca3d2af5d263fb,sentry-release=Unknown"},"correlation_id":"01KWFDMV5KSW3K5V73P18PBZ3F","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:21.793Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-835a1ce8c226dacc205efbb6: done: 0.166501 sec","job_status":"done","queue_duration_s":0.014459,"scheduling_latency_s":0.014481,"redis_calls":15,"redis_duration_s":0.012293,"redis_read_bytes":14,"redis_write_bytes":1727,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.004761,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":845,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005227,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002305,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":15,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001085892,"duration_s":0.166501,"completed_at":"2026-07-01T18:04:21.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 194.373069] nixbot[3045]: INFO: ::ffff:127.0.0.1:37876 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 194.401398] gitaly[1379]: time="2026-07-01T18:04:22.402Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMY287CZMXXPY9KBWT7VM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T18:04:52.092" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:22.392" grpc.time_ms=9.571 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 194.468961] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 74 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 194.805113] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:22 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 845 vm-test-run-nixbot-gitlab> gitlab # [ 194.806434] nixbot[3045]: {"ts": "2026-07-01T18:04:22.807Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 194.840112] nixbot[3045]: {"ts": "2026-07-01T18:04:22.841Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 194.891262] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 41 vm-test-run-nixbot-gitlab> gitlab # [ 194.894448] nixbot[3045]: {"ts": "2026-07-01T18:04:22.896Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 194.897139] nixbot[3045]: {"ts": "2026-07-01T18:04:22.898Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab # [ 195.051705] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 149 vm-test-run-nixbot-gitlab> gitlab # [ 195.060214] nixbot[3045]: {"ts": "2026-07-01T18:04:23.061Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.177585] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 102 vm-test-run-nixbot-gitlab> gitlab # [ 195.179510] nixbot[3045]: {"ts": "2026-07-01T18:04:23.181Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.237749] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 43 vm-test-run-nixbot-gitlab> gitlab # [ 195.239306] nixbot[3045]: {"ts": "2026-07-01T18:04:23.240Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.333316] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 91 vm-test-run-nixbot-gitlab> gitlab # [ 195.335090] nixbot[3045]: {"ts": "2026-07-01T18:04:23.336Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.400574] gitaly[1379]: time="2026-07-01T18:04:23.401Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZDKGD8HHBFQKGRVRCDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-01T18:04:53.091" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:23.392" grpc.time_ms=9.177 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 195.440639] gitaly[1379]: time="2026-07-01T18:04:23.417Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZDKGD8HHBFQKGRVRCDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:53.116" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:23.416" grpc.time_ms=0.538 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 195.461320] gitaly[1379]: time="2026-07-01T18:04:23.427Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZDKGD8HHBFQKGRVRCDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-01T18:04:33.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:23.420" grpc.time_ms=6.85 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 195.483803] nixbot[3045]: {"ts": "2026-07-01T18:04:23.477Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.488468] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 132 vm-test-run-nixbot-gitlab> gitlab # [ 195.533529] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 53 vm-test-run-nixbot-gitlab> gitlab # [ 195.537877] nixbot[3045]: {"ts": "2026-07-01T18:04:23.536Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 195.545130] nixbot[3045]: {"ts": "2026-07-01T18:04:23.546Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 195.555830] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 49 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds) vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-01T18:04:23.039Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.19 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 195.624344] gitaly[1379]: time="2026-07-01T18:04:23.626Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZMW2YSS3QGGAFVMX32P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.618" grpc.time_ms=7.994 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.626111] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 47 vm-test-run-nixbot-gitlab> gitlab # [ 195.651678] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 55 vm-test-run-nixbot-gitlab> gitlab # [ 195.714367] gitaly[1379]: time="2026-07-01T18:04:23.716Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZPJQKS5VZ71EES6XHE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.707" pid=1379 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.716646] gitaly[1379]: time="2026-07-01T18:04:23.717Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZPJQKS5VZ71EES6XHE4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.707" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.717841] gitaly[1379]: time="2026-07-01T18:04:23.717Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZPJQKS5VZ71EES6XHE4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.707" grpc.time_ms=10.286 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.719831] gitlab-workhorse[1279]: time="2026-07-01T18:04:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 vm-test-run-nixbot-gitlab> gitlab # [ 195.720765] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 86 vm-test-run-nixbot-gitlab> gitlab # [ 195.736692] gitaly[1379]: time="2026-07-01T18:04:23.738Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZQBRHXY8NQHV5F2YH10 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.726" grpc.time_ms=12.479 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 195.737937] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 80 vm-test-run-nixbot-gitlab> gitlab # [ 195.784198] gitaly[1379]: time="2026-07-01T18:04:23.786Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZSCBMEFJRH3P0WK0GE6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.770" pid=1379 remote_ip="2001:db8:1::1" request_sha=b63582508694cedf44c28c519477b093248f08cc response_bytes=373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.785435] gitaly[1379]: time="2026-07-01T18:04:23.786Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1234 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZSCBMEFJRH3P0WK0GE6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.770" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.787470] gitaly[1379]: time="2026-07-01T18:04:23.787Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1234 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZSCBMEFJRH3P0WK0GE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.770" grpc.time_ms=16.318 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.860944] gitlab-workhorse[1279]: time="2026-07-01T18:04:23Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=373 vm-test-run-nixbot-gitlab> gitlab # [ 195.862469] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 373 "" "git/2.54.0" 62 vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 195.968459] gitaly[1379]: time="2026-07-01T18:04:23.970Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZZD1REEZJVGBMCAKEZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.962" grpc.time_ms=7.608 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 195.971279] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:23 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 54 vm-test-run-nixbot-gitlab> gitlab # [ 196.061275] gitaly[1379]: time="2026-07-01T18:04:24.063Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN01X2D14P1SC6GZ5VRDJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:24.041" pid=1379 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 196.061967] gitaly[1379]: time="2026-07-01T18:04:24.063Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN01X2D14P1SC6GZ5VRDJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:24.041" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 196.065452] gitlab-workhorse[1279]: time="2026-07-01T18:04:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 196.066320] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 69 vm-test-run-nixbot-gitlab> gitlab # [ 196.067212] gitaly[1379]: time="2026-07-01T18:04:24.068Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN01X2D14P1SC6GZ5VRDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:24.041" grpc.time_ms=25.036 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 196.093098] gitaly[1379]: time="2026-07-01T18:04:24.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-01T18:04:34.007" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.087" grpc.time_ms=6.735 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.123264] gitaly[1379]: time="2026-07-01T18:04:24.125Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-01T18:04:54.018" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-01T18:04:24.118" grpc.time_ms=6.509 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.132676] gitaly[1379]: time="2026-07-01T18:04:24.134Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W duration_ms=94ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 196.152706] gitaly[1379]: time="2026-07-01T18:04:24.154Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W duration_ms=18ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 196.154296] gitaly[1379]: time="2026-07-01T18:04:24.156Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.039" grpc.time_ms=116.633 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.232173] gitaly[1379]: time="2026-07-01T18:04:24.234Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.233" grpc.time_ms=0.455 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.278130] gitaly[1379]: time="2026-07-01T18:04:24.278Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.277" grpc.time_ms=0.485 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.285238] gitaly[1379]: time="2026-07-01T18:04:24.287Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:54.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.285" grpc.time_ms=0.952 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.339603] gitaly[1379]: time="2026-07-01T18:04:24.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.341" grpc.time_ms=0.34 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.367510] gitaly[1379]: time="2026-07-01T18:04:24.369Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-01T18:04:54.062" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.362" grpc.time_ms=6.549 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.400099] gitaly[1379]: time="2026-07-01T18:04:24.402Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:34.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.395" grpc.time_ms=6.285 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.427651] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.428Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","efb675e1dce75529c90e19587e3d54bc59651543 6bc363e76e16113eddc459ed4c9872dfa5b0cac0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7e242d4df4d71617fdb756c9","created_at":"2026-07-01T18:04:24.407Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-701711ed8ef5486b","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.410Z","job_size_bytes":185,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7e242d4df4d71617fdb756c9: start","job_status":"start","queue_duration_s":0.017854,"scheduling_latency_s":0.017889} vm-test-run-nixbot-gitlab> gitlab # [ 196.455100] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.441Z","message":"changes: \"efb675e1dce75529c90e19587e3d54bc59651543 6bc363e76e16113eddc459ed4c9872dfa5b0cac0 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 196.459731] gitaly[1379]: time="2026-07-01T18:04:24.436Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W duration_ms=53ns method=POST pid=1379 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 196.465201] gitaly[1379]: time="2026-07-01T18:04:24.436Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.381" grpc.time_ms=54.704 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.482292] gitaly[1379]: time="2026-07-01T18:04:24.467Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:34.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.455" grpc.time_ms=11.285 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 196.503799] gitaly[1379]: time="2026-07-01T18:04:24.469Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W diskcache=53fb0e17-2d1a-47a0-892d-9ba162de59bd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.980" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # efb675e..6bc363e master -> master[ 196.505442] gitlab-workhorse[1279]: time="2026-07-01T18:04:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 196.507256] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:24 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 579 vm-test-run-nixbot-gitlab> gitlab # [ 196.508435] gitaly[1379]: time="2026-07-01T18:04:24.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=161 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=880640 command.minflt=13110 command.oublock=32 command.real_time_ms=479 command.spawn_token_fork_ms=0 command.system_time_ms=119 command.user_time_ms=42 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:23.980" grpc.time_ms=490.954 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.510437] gitaly[1379]: time="2026-07-01T18:04:24.481Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T18:04:34.060" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:24.481" grpc.time_ms=0.254 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.512385] gitaly[1379]: time="2026-07-01T18:04:24.485Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-01T18:04:34.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:24.472" grpc.time_ms=12.913 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.516384] gitaly[1379]: time="2026-07-01T18:04:24.495Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T18:04:34.069" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.489" grpc.time_ms=5.369 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.538398] gitaly[1379]: time="2026-07-01T18:04:24.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-01T18:04:34.050" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.500" grpc.time_ms=6.509 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.609202] gitaly[1379]: time="2026-07-01T18:04:24.519Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-01T18:04:54.010" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.511" grpc.time_ms=8.365 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.610777] gitaly[1379]: time="2026-07-01T18:04:24.536Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-01T18:04:34.077" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-01T18:04:24.528" grpc.time_ms=8.728 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.611907] gitaly[1379]: time="2026-07-01T18:04:24.538Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.537" grpc.time_ms=0.346 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 196.680857] gitaly[1379]: time="2026-07-01T18:04:24.558Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:34.094" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.544" grpc.time_ms=13.88 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.682106] gitaly[1379]: time="2026-07-01T18:04:24.566Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:24.565" grpc.time_ms=0.362 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 196.683361] gitaly[1379]: time="2026-07-01T18:04:24.660Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN diskcache=67b8650f-61d9-48c7-a5e4-0000475d9898 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:34.076" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:24.496" pid=1379 ref=refs/keep-around/efb675e1dce75529c90e19587e3d54bc59651543 remote_ip="2001:db8:1::1" revision=efb675e1dce75529c90e19587e3d54bc59651543 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.684128] gitaly[1379]: time="2026-07-01T18:04:24.661Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=51 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5186 command.oublock=8 command.real_time_ms=153 command.spawn_token_fork_ms=2 command.system_time_ms=36 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:34.076" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:24.496" grpc.time_ms=164.222 pid=1379 ref=refs/keep-around/efb675e1dce75529c90e19587e3d54bc59651543 remote_ip="2001:db8:1::1" revision=efb675e1dce75529c90e19587e3d54bc59651543 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.762370] gitaly[1379]: time="2026-07-01T18:04:24.672Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:54.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.664" grpc.time_ms=0.946 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.763448] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.726Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","efb675e1dce75529c90e19587e3d54bc59651543 6bc363e76e16113eddc459ed4c9872dfa5b0cac0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7e242d4df4d71617fdb756c9","created_at":"2026-07-01T18:04:24.407Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-701711ed8ef5486b","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.410Z","job_size_bytes":185,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7e242d4df4d71617fdb756c9: done: 0.297852 sec","job_status":"done","queue_duration_s":0.017854,"scheduling_latency_s":0.017889,"gitaly_calls":7,"gitaly_duration_s":0.092551,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.021008,"redis_read_bytes":350,"redis_write_bytes":13068,"redis_cache_calls":9,"redis_cache_duration_s":0.002283,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001344,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6768,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001982,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007292,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007647,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.019,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.019,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.167191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000500343,"exclusive_lock_hold_duration_s":0.020747609,"duration_s":0.297852,"completed_at":"2026-07-01T18:04:24.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023243,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 196.863210] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.727Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7501ec0ef5e95ad1eb80fe1d","created_at":"2026-07-01T18:04:24.488Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.490Z","job_size_bytes":46,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7501ec0ef5e95ad1eb80fe1d: start","job_status":"start","queue_duration_s":0.236742,"scheduling_latency_s":0.236766} vm-test-run-nixbot-gitlab> gitlab # [ 196.864785] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.744Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7501ec0ef5e95ad1eb80fe1d","created_at":"2026-07-01T18:04:24.488Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.490Z","job_size_bytes":46,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7501ec0ef5e95ad1eb80fe1d: done: 0.017207 sec","job_status":"done","queue_duration_s":0.236742,"scheduling_latency_s":0.236766,"redis_calls":9,"redis_duration_s":0.002262,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000207,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017207,"completed_at":"2026-07-01T18:04:24.744Z","load_balancing_strategy":"primary","db_duration_s":0.001552,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 196.952252] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.746Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"404f7ba84a04514fd272009c","created_at":"2026-07-01T18:04:24.495Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.496Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-404f7ba84a04514fd272009c: start","job_status":"start","queue_duration_s":0.249344,"scheduling_latency_s":0.249365} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.45 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab # [ 196.953886] gitaly[1379]: time="2026-07-01T18:04:24.711Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-01T18:04:34.258" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.708" grpc.time_ms=3.183 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 196.954960] gitaly[1379]: time="2026-07-01T18:04:24.830Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W diskcache=2a53fd7b-169e-4b92-bc09-c1a966c79c7b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-02T00:04:24.760" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.760" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.003462] nixbot[3045]: {"ts": "2026-07-01T18:04:24.884Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 197.004870] nixbot[3045]: {"ts": "2026-07-01T18:04:24.891Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-efb675e1dce7-12eadb97", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 197.005986] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.858Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"404f7ba84a04514fd272009c","created_at":"2026-07-01T18:04:24.495Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.496Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-404f7ba84a04514fd272009c: done: 0.112529 sec","job_status":"done","queue_duration_s":0.249344,"scheduling_latency_s":0.249365,"gitaly_calls":1,"gitaly_duration_s":0.075686,"redis_calls":12,"redis_duration_s":0.002929,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000709,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.096696648,"duration_s":0.112529,"completed_at":"2026-07-01T18:04:24.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 197.079775] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:24.860Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"83ad70cba890c3b0450d3089","created_at":"2026-07-01T18:04:24.608Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f7c40858951d9ba281d4d2457d07ceb82b2efd979059d7d7a91349205931d0b3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.610Z","job_size_bytes":467,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-83ad70cba890c3b0450d3089: start","job_status":"start","queue_duration_s":0.250007,"scheduling_latency_s":0.250029} vm-test-run-nixbot-gitlab> gitlab # [ 197.081305] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 644 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.14 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 197.082300] gitaly[1379]: time="2026-07-01T18:04:24.831Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1645 command.oublock=0 command.real_time_ms=69 command.spawn_token_fork_ms=33 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-02T00:04:24.760" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.760" grpc.time_ms=69.684 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0' vm-test-run-nixbot-gitlab> gitlab # [ 197.083168] gitaly[1379]: time="2026-07-01T18:04:24.881Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:54.379" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:24.880" grpc.time_ms=0.694 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.176732] gitaly[1379]: time="2026-07-01T18:04:25.178Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=293 command.spawn_token_fork_ms=288 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T18:04:34.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:24.884" grpc.time_ms=293.913 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0', in 0.05 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 197.199593] nixbot[3045]: INFO: ::ffff:127.0.0.1:37878 - "GET /api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 197.224591] gitaly[1379]: time="2026-07-01T18:04:25.226Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:55.025" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:25.225" grpc.time_ms=0.909 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.231779] gitaly[1379]: time="2026-07-01T18:04:25.233Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=79 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-01T18:04:55.029" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:25.229" grpc.time_ms=3.291 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.270506] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 197.310126] nix-daemon[3333]: accepted connection from pid 3331, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 197.310503] gitaly[1379]: time="2026-07-01T18:04:25.312Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:35.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:25.305" grpc.time_ms=6.497 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.334699] nix-daemon[3333]: accepted connection from pid 3332, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 197.741234] gitaly[1379]: time="2026-07-01T18:04:25.742Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-01T18:04:55.520" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:25.721" grpc.time_ms=21.118 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.813119] gitaly[1379]: time="2026-07-01T18:04:25.814Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDMZYNAFHZBQQMRX8P8H5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-01T18:04:55.597" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:25.797" grpc.time_ms=17.047 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 197.838553] sidekiq-cluster[2103]: {"severity":"WARN","time":"2026-07-01T18:04:25.834Z","class":"CreatePipelineWorker","correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"efb675e1dce75529c90e19587e3d54bc59651543","after":"6bc363e76e16113eddc459ed4c9872dfa5b0cac0","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6bc363e76e16113eddc459ed4c9872dfa5b0cac0","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 197.843842] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:25.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"83ad70cba890c3b0450d3089","created_at":"2026-07-01T18:04:24.608Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f7c40858951d9ba281d4d2457d07ceb82b2efd979059d7d7a91349205931d0b3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.610Z","job_size_bytes":467,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-83ad70cba890c3b0450d3089: done: 0.984624 sec","job_status":"done","queue_duration_s":0.250007,"scheduling_latency_s":0.250029,"gitaly_calls":7,"gitaly_duration_s":0.473551,"redis_calls":33,"redis_duration_s":0.007472,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001159,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002269,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001793,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.461537,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.984624,"completed_at":"2026-07-01T18:04:25.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035874,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 197.855520] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:25.848Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"a8070edd-a047-4efe-a49a-de8c085af098\"}"],"class":"WebHookWorker","jid":"45af4d5ccebf21bf7c481b7c","created_at":"2026-07-01T18:04:24.695Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:caa56f85827c6df51f20559f0dcd5782dc961376bf865a18184daee3e136b475","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.699Z","job_size_bytes":1749,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-45af4d5ccebf21bf7c481b7c: start","job_status":"start","queue_duration_s":1.14877,"scheduling_latency_s":1.148823} vm-test-run-nixbot-gitlab> gitlab # [ 197.929610] nixbot[3045]: INFO: ::ffff:127.0.0.1:37884 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 197.958289] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:25.958Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"a8070edd-a047-4efe-a49a-de8c085af098\"}"],"class":"WebHookWorker","jid":"45af4d5ccebf21bf7c481b7c","created_at":"2026-07-01T18:04:24.695Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-8853279ebcd14c50","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:caa56f85827c6df51f20559f0dcd5782dc961376bf865a18184daee3e136b475","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.699Z","job_size_bytes":1749,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-45af4d5ccebf21bf7c481b7c: done: 0.110219 sec","job_status":"done","queue_duration_s":1.14877,"scheduling_latency_s":1.148823,"redis_calls":19,"redis_duration_s":0.008425,"redis_read_bytes":31,"redis_write_bytes":5281,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002351,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000551,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3643,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003973,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00155,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.011379938000004586,"cpu_s":0.074251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000811835,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.110219,"completed_at":"2026-07-01T18:04:25.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 197.969517] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:25.962Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a1531d064a77d068f843ddd4","created_at":"2026-07-01T18:04:24.669Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.704Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a1531d064a77d068f843ddd4: start","job_status":"start","queue_duration_s":1.257532,"scheduling_latency_s":1.257578} vm-test-run-nixbot-gitlab> gitlab # [ 198.042180] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.007Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a1531d064a77d068f843ddd4","created_at":"2026-07-01T18:04:24.669Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.704Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a1531d064a77d068f843ddd4: done: 0.045479 sec","job_status":"done","queue_duration_s":1.257532,"scheduling_latency_s":1.257578,"redis_calls":18,"redis_duration_s":0.00419,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002256,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000363454,"duration_s":0.045479,"completed_at":"2026-07-01T18:04:26.007Z","load_balancing_strategy":"primary","db_duration_s":0.009764,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 198.062978] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.008Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9627c010209eb9f7f90f9223","created_at":"2026-07-01T18:04:24.705Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.708Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9627c010209eb9f7f90f9223: start","job_status":"start","queue_duration_s":1.300146,"scheduling_latency_s":1.300167} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0' vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0', in 0.03 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 198.148454] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.061Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9627c010209eb9f7f90f9223","created_at":"2026-07-01T18:04:24.705Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.708Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9627c010209eb9f7f90f9223: done: 0.052387 sec","job_status":"done","queue_duration_s":1.300146,"scheduling_latency_s":1.300167,"redis_calls":15,"redis_duration_s":0.00348,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000717,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000354515,"duration_s":0.052387,"completed_at":"2026-07-01T18:04:26.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 198.163138] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.063Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3c3cb9f9f7ae5949df5abd47","created_at":"2026-07-01T18:04:24.798Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.803Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c3cb9f9f7ae5949df5abd47: start","job_status":"start","queue_duration_s":1.25938,"scheduling_latency_s":1.259403} vm-test-run-nixbot-gitlab> gitlab # [ 198.227810] nixbot[3045]: INFO: ::ffff:127.0.0.1:37896 - "GET /api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 198.234381] gitaly[1379]: time="2026-07-01T18:04:26.236Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:56.034" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:26.234" grpc.time_ms=1.701 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.289393] gitaly[1379]: time="2026-07-01T18:04:26.291Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=53 command.spawn_token_fork_ms=47 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN1Z5BY3GF9ZDWW1BWM7D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.235" grpc.time_ms=54.353 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.291949] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 335 vm-test-run-nixbot-gitlab> gitlab # [ 198.315243] gitaly[1379]: time="2026-07-01T18:04:26.317Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:56.115" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:26.315" grpc.time_ms=0.77 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.344251] gitaly[1379]: time="2026-07-01T18:04:26.346Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN09EGAEWX3AX2PG6QZZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:56.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:26.345" grpc.time_ms=0.876 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.376177] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.377Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3c3cb9f9f7ae5949df5abd47","created_at":"2026-07-01T18:04:24.798Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-3874c2e514174e3e","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:24.803Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3c3cb9f9f7ae5949df5abd47: done: 0.31478 sec","job_status":"done","queue_duration_s":1.25938,"scheduling_latency_s":1.259403,"gitaly_calls":3,"gitaly_duration_s":0.032997,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011137000000000001,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001747,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001452,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001788,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000289,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000528,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000312,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.117,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.104,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.151161,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.28897555399999997,"duration_s":0.31478,"completed_at":"2026-07-01T18:04:26.377Z","load_balancing_strategy":"primary","db_duration_s":0.212633,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 198.390872] gitaly[1379]: time="2026-07-01T18:04:26.390Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2B23KCZSS9J4RG5T6TF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.382" pid=1379 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.465195] gitlab-workhorse[1279]: time="2026-07-01T18:04:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 198.466731] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 53 vm-test-run-nixbot-gitlab> gitlab # [ 198.467939] gitaly[1379]: time="2026-07-01T18:04:26.390Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2B23KCZSS9J4RG5T6TF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.382" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.468822] gitaly[1379]: time="2026-07-01T18:04:26.390Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2B23KCZSS9J4RG5T6TF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.382" grpc.time_ms=7.794 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.530456] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.532Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","45af4d5ccebf21bf7c481b7c"],"class":"WebHooks::LogExecutionWorker","jid":"32e9c081fd542b00b35a5072","created_at":"2026-07-01T18:04:25.933Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-546633814c1a4d6e","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:daf4b05ba83da81b10421c684633cd4f1e4aa5cbc127b75ddcd9535fafd92493","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:25.943Z","job_size_bytes":2506,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-32e9c081fd542b00b35a5072: start","job_status":"start","queue_duration_s":0.588529,"scheduling_latency_s":0.588566} vm-test-run-nixbot-gitlab> gitlab # [ 198.595781] gitaly[1379]: time="2026-07-01T18:04:26.597Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2CSEASWAYBN0K6AQQ65 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.578" pid=1379 remote_ip="2001:db8:1::1" request_sha=e16c60e54b4db11bc22cf797dbf933ab800259c0 response_bytes=389 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.596404] gitaly[1379]: time="2026-07-01T18:04:26.597Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1201 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2CSEASWAYBN0K6AQQ65 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.578" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.599566] gitaly[1379]: time="2026-07-01T18:04:26.600Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1201 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2CSEASWAYBN0K6AQQ65 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:26.578" grpc.time_ms=20.816 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 198.600968] gitlab-workhorse[1279]: time="2026-07-01T18:04:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=389 vm-test-run-nixbot-gitlab> gitlab # [ 198.603459] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 389 "" "git/2.54.0" 207 vm-test-run-nixbot-gitlab> gitlab # [ 198.631843] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.633Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","45af4d5ccebf21bf7c481b7c"],"class":"WebHooks::LogExecutionWorker","jid":"32e9c081fd542b00b35a5072","created_at":"2026-07-01T18:04:25.933Z","trace_propagation_headers":{"sentry-trace":"55ddb44e0d7646f988226b3d19d2c717-546633814c1a4d6e","baggage":"sentry-trace_id=55ddb44e0d7646f988226b3d19d2c717,sentry-release=Unknown"},"correlation_id":"01KWFDMZYNAFHZBQQMRX8P8H5W","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:daf4b05ba83da81b10421c684633cd4f1e4aa5cbc127b75ddcd9535fafd92493","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:25.943Z","job_size_bytes":2506,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-32e9c081fd542b00b35a5072: done: 0.10146 sec","job_status":"done","queue_duration_s":0.588529,"scheduling_latency_s":0.588566,"redis_calls":10,"redis_duration_s":0.002527,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000544,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348368,"duration_s":0.10146,"completed_at":"2026-07-01T18:04:26.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 198.718382] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7df29c1c0d5013e029394d75","created_at":"2026-07-01T18:04:26.220Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-a32477f703b84e5f","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:26.224Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7df29c1c0d5013e029394d75: start","job_status":"start","queue_duration_s":0.410839,"scheduling_latency_s":0.410861} vm-test-run-nixbot-gitlab> gitlab # [ 198.719799] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:26.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7df29c1c0d5013e029394d75","created_at":"2026-07-01T18:04:26.220Z","trace_propagation_headers":{"sentry-trace":"6e0928c1d28342b994d8301f26362bbe-a32477f703b84e5f","baggage":"sentry-trace_id=6e0928c1d28342b994d8301f26362bbe,sentry-release=Unknown"},"correlation_id":"01KWFDN09EGAEWX3AX2PG6QZZN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:26.224Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7df29c1c0d5013e029394d75: done: 0.033487 sec","job_status":"done","queue_duration_s":0.410839,"scheduling_latency_s":0.410861,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396977,"duration_s":0.033487,"completed_at":"2026-07-01T18:04:26.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 198.828421] nixbot[3045]: {"ts": "2026-07-01T18:04:26.830Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} vm-test-run-nixbot-gitlab> gitlab # [ 198.834942] nix-daemon[3340]: unexpected Nix daemon error: error: interrupted by the user vm-test-run-nixbot-gitlab> gitlab # [ 198.916878] gitaly[1379]: time="2026-07-01T18:04:26.917Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:56.615" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:26.916" grpc.time_ms=0.95 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.957514] gitaly[1379]: time="2026-07-01T18:04:26.959Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:56.858" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:26.958" grpc.time_ms=0.601 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.978485] gitaly[1379]: time="2026-07-01T18:04:26.979Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:36.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:26.952" grpc.time_ms=27.043 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 198.984487] gitaly[1379]: time="2026-07-01T18:04:26.986Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-01T18:04:56.879" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:26.979" grpc.time_ms=6.429 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.042471] gitaly[1379]: time="2026-07-01T18:04:27.044Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:37.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:27.036" grpc.time_ms=7.498 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.064357] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.058Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f5a0cae81908917f076792ea","created_at":"2026-07-01T18:04:27.053Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-0dc3e39bf2bc4d6a","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.056Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5a0cae81908917f076792ea: start","job_status":"start","queue_duration_s":0.001887,"scheduling_latency_s":0.002057} vm-test-run-nixbot-gitlab> gitlab # [ 199.089574] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 217 vm-test-run-nixbot-gitlab> gitlab # [ 199.090125] nixbot[3045]: {"ts": "2026-07-01T18:04:27.073Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 199.092122] gitaly[1379]: time="2026-07-01T18:04:27.094Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-01T18:04:37.011" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:27.093" grpc.time_ms=0.217 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.123596] gitaly[1379]: time="2026-07-01T18:04:27.125Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=18 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-01T18:04:37.020" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:27.100" grpc.time_ms=24.292 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.145470] gitaly[1379]: time="2026-07-01T18:04:27.132Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN32CBQJ8J1YK5YYYC3F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.129" grpc.time_ms=2.056 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.168575] gitaly[1379]: time="2026-07-01T18:04:27.170Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN32CBQJ8J1YK5YYYC3F2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:37.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:27.163" grpc.time_ms=6.728 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.172267] gitaly[1379]: time="2026-07-01T18:04:27.174Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:27.173" grpc.time_ms=0.831 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 199.208235] nixbot[3045]: {"ts": "2026-07-01T18:04:27.210Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 199.209170] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/efb675e1dce75529c90e19587e3d54bc59651543 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 125 vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0' vm-test-run-nixbot-gitlab> gitlab # [ 199.220655] gitaly[1379]: time="2026-07-01T18:04:27.222Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:27.222" grpc.time_ms=0.478 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 199.244416] gitaly[1379]: time="2026-07-01T18:04:27.246Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4158 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:27.245" grpc.time_ms=0.407 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 199.262254] gitaly[1379]: time="2026-07-01T18:04:27.263Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:27.261" grpc.time_ms=0.605 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0', in 0.05 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-01T18:04:26.756727Z', 'started_at': None, 'finished_at': None}] vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.72 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 199.273170] gitaly[1379]: time="2026-07-01T18:04:27.275Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH diskcache=1fc26859-4d85-4fc0-8b0f-43d728d4e92d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:37.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:27.128" pid=1379 ref=refs/keep-around/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 remote_ip="2001:db8:1::1" revision=6bc363e76e16113eddc459ed4c9872dfa5b0cac0 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.274349] gitaly[1379]: time="2026-07-01T18:04:27.275Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=105 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5088 command.oublock=8 command.real_time_ms=133 command.spawn_token_fork_ms=2 command.system_time_ms=88 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-01T18:04:37.047" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-01T18:04:27.128" grpc.time_ms=146.82 pid=1379 ref=refs/keep-around/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 remote_ip="2001:db8:1::1" revision=6bc363e76e16113eddc459ed4c9872dfa5b0cac0 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.278523] gitaly[1379]: time="2026-07-01T18:04:27.280Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.178" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.279" grpc.time_ms=1.157 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.282348] nixbot[3045]: INFO: ::ffff:127.0.0.1:37908 - "GET /api/repos/gitlab/root/test-flake/builds?commit=6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.08 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 199.340545] gitaly[1379]: time="2026-07-01T18:04:27.342Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN396J4AJDPHMKR8XVE7F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.241" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.341" grpc.time_ms=0.659 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.357221] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 55 vm-test-run-nixbot-gitlab> gitlab # [ 199.389715] gitaly[1379]: time="2026-07-01T18:04:27.391Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:27.391" grpc.time_ms=0.53 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 199.401890] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 570 vm-test-run-nixbot-gitlab> gitlab # [ 199.403086] nixbot[3045]: {"ts": "2026-07-01T18:04:27.404Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 199.409688] nixbot[3045]: {"ts": "2026-07-01T18:04:27.411Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6bc363e76e16-2b594550", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 199.428856] gitaly[1379]: time="2026-07-01T18:04:27.430Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E diskcache=00019396-9a85-4865-aa15-9a3fec0dd07e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:04:57.053" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:27.154" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.429498] gitaly[1379]: time="2026-07-01T18:04:27.431Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=115 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=827392 command.minflt=7090 command.oublock=0 command.real_time_ms=267 command.spawn_token_fork_ms=0 command.system_time_ms=87 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:04:57.053" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:27.154" grpc.time_ms=276.793 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.488895] nix-daemon[3333]: accepted connection from pid 3456, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 199.490265] gitaly[1379]: time="2026-07-01T18:04:27.491Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:57.390" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.490" grpc.time_ms=0.868 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.497098] nix-daemon[3333]: accepted connection from pid 3455, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 199.513600] gitaly[1379]: time="2026-07-01T18:04:27.515Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.414" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.514" grpc.time_ms=0.855 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.542374] gitaly[1379]: time="2026-07-01T18:04:27.544Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.442" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.543" grpc.time_ms=0.635 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 199.548207] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.550Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a69d2db4b96f45ff3cfc9462","created_at":"2026-07-01T18:04:27.548Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"f5a0cae81908917f076792ea","job_size_bytes":3,"pid":2103,"job_status":"deduplicated","message":"PipelineProcessWorker JID-a69d2db4b96f45ff3cfc9462: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 199.549920] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.551Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f5a0cae81908917f076792ea","created_at":"2026-07-01T18:04:27.053Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-0dc3e39bf2bc4d6a","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.056Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2103,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f5a0cae81908917f076792ea: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 199.555331] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.557Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f5a0cae81908917f076792ea","created_at":"2026-07-01T18:04:27.053Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-0dc3e39bf2bc4d6a","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.056Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5a0cae81908917f076792ea: done: 0.499017 sec","job_status":"done","queue_duration_s":0.001887,"scheduling_latency_s":0.002057,"gitaly_calls":4,"gitaly_duration_s":0.289077,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.028441,"redis_read_bytes":267,"redis_write_bytes":24008,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001161,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.00217,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.009195,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.003184,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12387,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.00574,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000218,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000591,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005021,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137705,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.456440413,"duration_s":0.499017,"completed_at":"2026-07-01T18:04:27.557Z","load_balancing_strategy":"primary","db_duration_s":0.045086,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 199.627719] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.558Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c92caa53313780bd2240d1af","created_at":"2026-07-01T18:04:27.135Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.141Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c92caa53313780bd2240d1af: start","job_status":"start","queue_duration_s":0.416648,"scheduling_latency_s":0.41667} vm-test-run-nixbot-gitlab> gitlab # [ 199.649806] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.643Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c92caa53313780bd2240d1af","created_at":"2026-07-01T18:04:27.135Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.141Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c92caa53313780bd2240d1af: done: 0.08519 sec","job_status":"done","queue_duration_s":0.416648,"scheduling_latency_s":0.41667,"redis_calls":15,"redis_duration_s":0.005065,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000635,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002395,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000872,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064106,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000602311,"duration_s":0.08519,"completed_at":"2026-07-01T18:04:27.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013039,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 199.727733] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.646Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"3ba6dc105a6e4ab056c659ff","created_at":"2026-07-01T18:04:27.634Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-8f465407956c43d2","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:0ff290da45a902dfae27f50398845f374a99339fa91898898a701ca63bc2cbd9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.636Z","job_size_bytes":464,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-3ba6dc105a6e4ab056c659ff: start","job_status":"start","queue_duration_s":0.00925,"scheduling_latency_s":0.009271} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 199.754307] sidekiq-cluster[2103]: {"severity":"WARN","time":"2026-07-01T18:04:27.684Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"3ba6dc105a6e4ab056c659ff","created_at":"2026-07-01T18:04:27.634Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-8f465407956c43d2","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:0ff290da45a902dfae27f50398845f374a99339fa91898898a701ca63bc2cbd9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.636Z","job_size_bytes":464,"pid":2103,"sidekiq_tid":"m44v","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-3ba6dc105a6e4ab056c659ff: fail: 0.037506 sec","job_status":"fail","queue_duration_s":0.00925,"scheduling_latency_s":0.009271,"redis_calls":7,"redis_duration_s":0.0029439999999999996,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037506,"completed_at":"2026-07-01T18:04:27.683Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.009085,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.10 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 200.403784] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.688Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b054d006efe0ab482e22844b","created_at":"2026-07-01T18:04:27.286Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.290Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b054d006efe0ab482e22844b: start","job_status":"start","queue_duration_s":0.398026,"scheduling_latency_s":0.398094} vm-test-run-nixbot-gitlab> gitlab # [ 200.405072] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.727Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b054d006efe0ab482e22844b","created_at":"2026-07-01T18:04:27.286Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.290Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b054d006efe0ab482e22844b: done: 0.039332 sec","job_status":"done","queue_duration_s":0.398026,"scheduling_latency_s":0.398094,"redis_calls":7,"redis_duration_s":0.003391,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039332,"completed_at":"2026-07-01T18:04:27.727Z","load_balancing_strategy":"primary","db_duration_s":0.006299,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 200.474126] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.729Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b36a515fbac7a3f858c593d4","created_at":"2026-07-01T18:04:27.290Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.293Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b36a515fbac7a3f858c593d4: start","job_status":"start","queue_duration_s":0.436515,"scheduling_latency_s":0.436541} vm-test-run-nixbot-gitlab> gitlab # [ 200.496547] gitaly[1379]: time="2026-07-01T18:04:27.847Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:57.745" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.846" grpc.time_ms=1.142 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 200.516780] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 64 vm-test-run-nixbot-gitlab> gitlab # [ 200.520951] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.783Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b36a515fbac7a3f858c593d4","created_at":"2026-07-01T18:04:27.290Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.293Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b36a515fbac7a3f858c593d4: done: 0.053831 sec","job_status":"done","queue_duration_s":0.436515,"scheduling_latency_s":0.436541,"redis_calls":10,"redis_duration_s":0.004477,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053831,"completed_at":"2026-07-01T18:04:27.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 200.641960] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.785Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5916b9e0837bd1a089d0be15","created_at":"2026-07-01T18:04:27.387Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.390Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5916b9e0837bd1a089d0be15: start","job_status":"start","queue_duration_s":0.394654,"scheduling_latency_s":0.394683} vm-test-run-nixbot-gitlab> gitlab # [ 200.643050] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.926Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5916b9e0837bd1a089d0be15","created_at":"2026-07-01T18:04:27.387Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-04fad8cf4ce242d0","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.390Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5916b9e0837bd1a089d0be15: done: 0.141322 sec","job_status":"done","queue_duration_s":0.394654,"scheduling_latency_s":0.394683,"gitaly_calls":3,"gitaly_duration_s":0.010859,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.012938,"redis_read_bytes":99,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001528,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003352,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000673,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002786,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000497,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000868,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001706,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083103,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.110911303,"duration_s":0.141322,"completed_at":"2026-07-01T18:04:27.926Z","load_balancing_strategy":"primary","db_duration_s":0.040821,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 200.743158] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"13f34398e503fc16bd7e94d8","created_at":"2026-07-01T18:04:27.435Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.440Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-13f34398e503fc16bd7e94d8: start","job_status":"start","queue_duration_s":0.489296,"scheduling_latency_s":0.489338} vm-test-run-nixbot-gitlab> gitlab # [ 200.744407] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.960Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"13f34398e503fc16bd7e94d8","created_at":"2026-07-01T18:04:27.435Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.440Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-13f34398e503fc16bd7e94d8: done: 0.03084 sec","job_status":"done","queue_duration_s":0.489296,"scheduling_latency_s":0.489338,"redis_calls":10,"redis_duration_s":0.002721,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386083,"duration_s":0.03084,"completed_at":"2026-07-01T18:04:27.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 200.848219] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"04b52fc510f988f44cc341b5","created_at":"2026-07-01T18:04:27.450Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.454Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-04b52fc510f988f44cc341b5: start","job_status":"start","queue_duration_s":0.507456,"scheduling_latency_s":0.507478} vm-test-run-nixbot-gitlab> gitlab # [ 200.849776] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.987Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"04b52fc510f988f44cc341b5","created_at":"2026-07-01T18:04:27.450Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.454Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-04b52fc510f988f44cc341b5: done: 0.02589 sec","job_status":"done","queue_duration_s":0.507456,"scheduling_latency_s":0.507478,"redis_calls":10,"redis_duration_s":0.002198,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334959,"duration_s":0.02589,"completed_at":"2026-07-01T18:04:27.987Z","load_balancing_strategy":"primary","db_duration_s":0.004399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 200.928428] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:27.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"69641d616b656c072ded9367","created_at":"2026-07-01T18:04:27.463Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.466Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-69641d616b656c072ded9367: start","job_status":"start","queue_duration_s":0.522248,"scheduling_latency_s":0.522269} vm-test-run-nixbot-gitlab> gitlab # [ 200.929748] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"69641d616b656c072ded9367","created_at":"2026-07-01T18:04:27.463Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.466Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-69641d616b656c072ded9367: done: 0.025907 sec","job_status":"done","queue_duration_s":0.522248,"scheduling_latency_s":0.522269,"redis_calls":13,"redis_duration_s":0.003226,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000907,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000358984,"duration_s":0.025907,"completed_at":"2026-07-01T18:04:28.014Z","load_balancing_strategy":"primary","db_duration_s":0.003258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 200.983304] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.016Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"53baa3a0cd9551e7600c6603","created_at":"2026-07-01T18:04:27.467Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.472Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-53baa3a0cd9551e7600c6603: start","job_status":"start","queue_duration_s":0.543848,"scheduling_latency_s":0.543887} vm-test-run-nixbot-gitlab> gitlab # [ 201.060994] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"53baa3a0cd9551e7600c6603","created_at":"2026-07-01T18:04:27.467Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.472Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-53baa3a0cd9551e7600c6603: done: 0.022888 sec","job_status":"done","queue_duration_s":0.543848,"scheduling_latency_s":0.543887,"redis_calls":10,"redis_duration_s":0.0021780000000000002,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361778,"duration_s":0.022888,"completed_at":"2026-07-01T18:04:28.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.079792] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8eb196c6426571c4765e262c","created_at":"2026-07-01T18:04:27.473Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.477Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8eb196c6426571c4765e262c: start","job_status":"start","queue_duration_s":0.56359,"scheduling_latency_s":0.56361} vm-test-run-nixbot-gitlab> gitlab # [ 201.167313] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8eb196c6426571c4765e262c","created_at":"2026-07-01T18:04:27.473Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.477Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8eb196c6426571c4765e262c: done: 0.112487 sec","job_status":"done","queue_duration_s":0.56359,"scheduling_latency_s":0.56361,"redis_calls":11,"redis_duration_s":0.003006,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002087,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.089,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.087,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352,"duration_s":0.112487,"completed_at":"2026-07-01T18:04:28.153Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.174763] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.154Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5f955cf12dfaaeebdd716211","created_at":"2026-07-01T18:04:27.478Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.483Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5f955cf12dfaaeebdd716211: start","job_status":"start","queue_duration_s":0.671202,"scheduling_latency_s":0.671223} vm-test-run-nixbot-gitlab> gitlab # [ 201.202255] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.176Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5f955cf12dfaaeebdd716211","created_at":"2026-07-01T18:04:27.478Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.483Z","job_size_bytes":65,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5f955cf12dfaaeebdd716211: done: 0.022187 sec","job_status":"done","queue_duration_s":0.671202,"scheduling_latency_s":0.671223,"redis_calls":10,"redis_duration_s":0.0022,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331327,"duration_s":0.022187,"completed_at":"2026-07-01T18:04:28.176Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.311428] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1fcb964ddc500821961d33ef","created_at":"2026-07-01T18:04:27.484Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.485Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1fcb964ddc500821961d33ef: start","job_status":"start","queue_duration_s":0.692308,"scheduling_latency_s":0.692328} vm-test-run-nixbot-gitlab> gitlab # [ 201.312752] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.200Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"1fcb964ddc500821961d33ef","created_at":"2026-07-01T18:04:27.484Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.485Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-1fcb964ddc500821961d33ef: done: 0.022461 sec","job_status":"done","queue_duration_s":0.692308,"scheduling_latency_s":0.692328,"redis_calls":7,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022461,"completed_at":"2026-07-01T18:04:28.200Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 201.384231] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.201Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c5cad97a0f00afd314ac4816","created_at":"2026-07-01T18:04:27.516Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.520Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c5cad97a0f00afd314ac4816: start","job_status":"start","queue_duration_s":0.681243,"scheduling_latency_s":0.681265} vm-test-run-nixbot-gitlab> gitlab # [ 201.453062] gitaly[1379]: time="2026-07-01T18:04:27.879Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.777" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.878" grpc.time_ms=1.084 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.08 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 201.454627] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.227Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c5cad97a0f00afd314ac4816","created_at":"2026-07-01T18:04:27.516Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.520Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c5cad97a0f00afd314ac4816: done: 0.025881 sec","job_status":"done","queue_duration_s":0.681243,"scheduling_latency_s":0.681265,"redis_calls":10,"redis_duration_s":0.00231,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"duration_s":0.025881,"completed_at":"2026-07-01T18:04:28.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.523899] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4fa33c26bbad6c38b9a694a4","created_at":"2026-07-01T18:04:27.521Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.524Z","job_size_bytes":6,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4fa33c26bbad6c38b9a694a4: start","job_status":"start","queue_duration_s":0.704836,"scheduling_latency_s":0.704856} vm-test-run-nixbot-gitlab> gitlab # [ 201.525245] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.247Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4fa33c26bbad6c38b9a694a4","created_at":"2026-07-01T18:04:27.521Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.524Z","job_size_bytes":6,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4fa33c26bbad6c38b9a694a4: done: 0.018212 sec","job_status":"done","queue_duration_s":0.704836,"scheduling_latency_s":0.704856,"redis_calls":10,"redis_duration_s":0.002314,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000364851,"duration_s":0.018212,"completed_at":"2026-07-01T18:04:28.247Z","load_balancing_strategy":"primary","db_duration_s":0.001549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.596081] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.248Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"59a4b1a718a89ec56282167b","created_at":"2026-07-01T18:04:27.552Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.553Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-59a4b1a718a89ec56282167b: start","job_status":"start","queue_duration_s":0.695667,"scheduling_latency_s":0.695687} vm-test-run-nixbot-gitlab> gitlab # [ 201.599335] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.314Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"59a4b1a718a89ec56282167b","created_at":"2026-07-01T18:04:27.552Z","trace_propagation_headers":{"sentry-trace":"1250c54a92284c2bb73a9eca5e42735f-b96e16b1ba8740ce","baggage":"sentry-trace_id=1250c54a92284c2bb73a9eca5e42735f,sentry-release=Unknown"},"correlation_id":"01KWFDN2V591BVY2QVQH2ZXH2E","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.553Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-59a4b1a718a89ec56282167b: done: 0.065238 sec","job_status":"done","queue_duration_s":0.695667,"scheduling_latency_s":0.695687,"gitaly_calls":2,"gitaly_duration_s":0.00589,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003206,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000609,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001405,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000217,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.046126253,"duration_s":0.065238,"completed_at":"2026-07-01T18:04:28.313Z","load_balancing_strategy":"primary","db_duration_s":0.012728,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 201.693318] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.315Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c4725d029ab3f63ce503081c","created_at":"2026-07-01T18:04:27.832Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-d342d336cd9143a3","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.835Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c4725d029ab3f63ce503081c: start","job_status":"start","queue_duration_s":0.479452,"scheduling_latency_s":0.479473} vm-test-run-nixbot-gitlab> gitlab # [ 201.697807] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:28.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c4725d029ab3f63ce503081c","created_at":"2026-07-01T18:04:27.832Z","trace_propagation_headers":{"sentry-trace":"d466cd4357cf4a9f99ac064f18d9b96d-d342d336cd9143a3","baggage":"sentry-trace_id=d466cd4357cf4a9f99ac064f18d9b96d,sentry-release=Unknown"},"correlation_id":"01KWFDN2THDFPGRFZ8EJWTFZKH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:27.835Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c4725d029ab3f63ce503081c: done: 0.022648 sec","job_status":"done","queue_duration_s":0.479452,"scheduling_latency_s":0.479473,"redis_calls":7,"redis_duration_s":0.001607,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022648,"completed_at":"2026-07-01T18:04:28.338Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 201.785346] gitaly[1379]: time="2026-07-01T18:04:27.908Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2THDFPGRFZ8EJWTFZKH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:57.805" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:27.906" grpc.time_ms=1.092 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.786707] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 49 vm-test-run-nixbot-gitlab> gitlab # [ 201.788148] gitaly[1379]: time="2026-07-01T18:04:28.278Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:04:58.177" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:28.277" grpc.time_ms=0.783 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.857363] gitaly[1379]: time="2026-07-01T18:04:28.305Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN2V591BVY2QVQH2ZXH2E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:58.203" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:28.304" grpc.time_ms=0.598 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.858925] gitaly[1379]: time="2026-07-01T18:04:28.426Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN4B0BKMW74B6HRN5VQ96 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:58.325" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:28.426" grpc.time_ms=0.597 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 201.860107] gitaly[1379]: time="2026-07-01T18:04:29.523Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN5DGPVZEX8E5V75ESFD4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:04:59.422" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:29.522" grpc.time_ms=0.543 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 202.663701] gitaly[1379]: time="2026-07-01T18:04:30.665Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN6FWD1VPXZDB00YY8110 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:00.562" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:30.663" grpc.time_ms=1.327 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 202.749218] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 161 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.22 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 203.892655] gitaly[1379]: time="2026-07-01T18:04:31.893Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN7NYH45TEXT83M3YF9YC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:01.790" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:31.891" grpc.time_ms=1.321 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 203.960837] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 155 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.21 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 204.222889] nix-daemon[3333]: accepted connection from pid 3490, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 204.254192] gitaly[1379]: time="2026-07-01T18:04:32.256Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:02.153" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:32.254" grpc.time_ms=0.986 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.344664] gitaly[1379]: time="2026-07-01T18:04:32.346Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:42.170" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:32.331" grpc.time_ms=14.678 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.385931] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:32.386Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"62b871c078f3769510c8549e","created_at":"2026-07-01T18:04:32.380Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-af4822fd34704a58","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.383Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-62b871c078f3769510c8549e: start","job_status":"start","queue_duration_s":0.002588,"scheduling_latency_s":0.002656} vm-test-run-nixbot-gitlab> gitlab # [ 204.432891] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 243 vm-test-run-nixbot-gitlab> gitlab # [ 204.440308] nixbot[3045]: {"ts": "2026-07-01T18:04:32.440Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 204.516367] gitaly[1379]: time="2026-07-01T18:04:32.518Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN8A02YP3SMRARK8FZ5H1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:02.416" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:32.516" grpc.time_ms=0.913 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.551055] gitaly[1379]: time="2026-07-01T18:04:32.552Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN8A02YP3SMRARK8FZ5H1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:42.224" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:32.544" grpc.time_ms=8.351 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.578482] gitaly[1379]: time="2026-07-01T18:04:32.580Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:32.580" grpc.time_ms=0.389 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 204.603814] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 156 vm-test-run-nixbot-gitlab> gitlab # [ 204.608209] nixbot[3045]: {"ts": "2026-07-01T18:04:32.609Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 204.619118] gitaly[1379]: time="2026-07-01T18:04:32.621Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:32.620" grpc.time_ms=0.545 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 204.660043] gitaly[1379]: time="2026-07-01T18:04:32.662Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:32.661" grpc.time_ms=0.318 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 204.684610] gitaly[1379]: time="2026-07-01T18:04:32.686Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB diskcache=d2083d9c-4c77-4b76-8f73-355bba8f31ad grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:02.404" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:32.505" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.685276] gitaly[1379]: time="2026-07-01T18:04:32.686Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=143 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7197 command.oublock=0 command.real_time_ms=174 command.spawn_token_fork_ms=0 command.system_time_ms=100 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:02.404" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:32.505" grpc.time_ms=181.69 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.787325] gitaly[1379]: time="2026-07-01T18:04:32.788Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:05:02.686" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:32.787" grpc.time_ms=1.371 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.826820] gitaly[1379]: time="2026-07-01T18:04:32.828Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:02.726" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:32.827" grpc.time_ms=0.877 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.900058] gitaly[1379]: time="2026-07-01T18:04:32.901Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:02.799" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:32.899" grpc.time_ms=1.269 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 204.918120] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:32.919Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f20d4a99abf8dc3738322484","created_at":"2026-07-01T18:04:32.915Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"62b871c078f3769510c8549e","job_size_bytes":3,"pid":2103,"job_status":"deduplicated","message":"PipelineProcessWorker JID-f20d4a99abf8dc3738322484: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 204.921983] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:32.923Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"62b871c078f3769510c8549e","created_at":"2026-07-01T18:04:32.380Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-af4822fd34704a58","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.383Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2103,"job_status":"rescheduled","message":"PipelineProcessWorker JID-62b871c078f3769510c8549e: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 204.934886] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:32.936Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"62b871c078f3769510c8549e","created_at":"2026-07-01T18:04:32.380Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-af4822fd34704a58","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.383Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-62b871c078f3769510c8549e: done: 0.549905 sec","job_status":"done","queue_duration_s":0.002588,"scheduling_latency_s":0.002656,"gitaly_calls":4,"gitaly_duration_s":0.264682,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.039869,"redis_read_bytes":262,"redis_write_bytes":23385,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002556,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.004018,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002274,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.008439,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.013097,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000657,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001649,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.004623,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.019,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.191596,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.487634042,"duration_s":0.549905,"completed_at":"2026-07-01T18:04:32.936Z","load_balancing_strategy":"primary","db_duration_s":0.063747,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 205.013151] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:32.939Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"86309660017b6a2f41fa75cb","created_at":"2026-07-01T18:04:32.496Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.498Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-86309660017b6a2f41fa75cb: start","job_status":"start","queue_duration_s":0.440924,"scheduling_latency_s":0.440977} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 205.019799] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.000Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"86309660017b6a2f41fa75cb","created_at":"2026-07-01T18:04:32.496Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.498Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-86309660017b6a2f41fa75cb: done: 0.061133 sec","job_status":"done","queue_duration_s":0.440924,"scheduling_latency_s":0.440977,"redis_calls":7,"redis_duration_s":0.00482,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061133,"completed_at":"2026-07-01T18:04:33.000Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010899,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 205.090360] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.002Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"07b50c4e8f1920e5f23a3d87","created_at":"2026-07-01T18:04:32.727Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.729Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-07b50c4e8f1920e5f23a3d87: start","job_status":"start","queue_duration_s":0.272538,"scheduling_latency_s":0.272564} vm-test-run-nixbot-gitlab> gitlab # [ 205.114251] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.046Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"07b50c4e8f1920e5f23a3d87","created_at":"2026-07-01T18:04:32.727Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.729Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-07b50c4e8f1920e5f23a3d87: done: 0.04451 sec","job_status":"done","queue_duration_s":0.272538,"scheduling_latency_s":0.272564,"redis_calls":7,"redis_duration_s":0.001675,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04451,"completed_at":"2026-07-01T18:04:33.046Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.198194] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.048Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c869f0f6c9310d057d3e88f0","created_at":"2026-07-01T18:04:32.742Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.746Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c869f0f6c9310d057d3e88f0: start","job_status":"start","queue_duration_s":0.301328,"scheduling_latency_s":0.301351} vm-test-run-nixbot-gitlab> gitlab # [ 205.270276] gitaly[1379]: time="2026-07-01T18:04:33.089Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN8V4EDPAVRZX5F6ZWM9B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:03.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:33.089" grpc.time_ms=0.748 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 205.272229] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 151 vm-test-run-nixbot-gitlab> gitlab # [ 205.273267] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c869f0f6c9310d057d3e88f0","created_at":"2026-07-01T18:04:32.742Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.746Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c869f0f6c9310d057d3e88f0: done: 0.166022 sec","job_status":"done","queue_duration_s":0.301328,"scheduling_latency_s":0.301351,"redis_calls":7,"redis_duration_s":0.001649,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001175,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.145,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.145,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.166022,"completed_at":"2026-07-01T18:04:33.214Z","load_balancing_strategy":"primary","db_duration_s":0.145248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.350888] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.215Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0056dd1d401c0064bde0ca9c","created_at":"2026-07-01T18:04:32.759Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.762Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0056dd1d401c0064bde0ca9c: start","job_status":"start","queue_duration_s":0.453277,"scheduling_latency_s":0.453302} vm-test-run-nixbot-gitlab> gitlab # [ 205.352211] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"0056dd1d401c0064bde0ca9c","created_at":"2026-07-01T18:04:32.759Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.762Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0056dd1d401c0064bde0ca9c: done: 0.02175 sec","job_status":"done","queue_duration_s":0.453277,"scheduling_latency_s":0.453302,"redis_calls":10,"redis_duration_s":0.002111,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00061,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02175,"completed_at":"2026-07-01T18:04:33.237Z","load_balancing_strategy":"primary","db_duration_s":0.00277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.433535] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"66c820ea27fecf7ffb9799df","created_at":"2026-07-01T18:04:32.763Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.765Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-66c820ea27fecf7ffb9799df: start","job_status":"start","queue_duration_s":0.472986,"scheduling_latency_s":0.473009} vm-test-run-nixbot-gitlab> gitlab # [ 205.435163] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.255Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"66c820ea27fecf7ffb9799df","created_at":"2026-07-01T18:04:32.763Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.765Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-66c820ea27fecf7ffb9799df: done: 0.017235 sec","job_status":"done","queue_duration_s":0.472986,"scheduling_latency_s":0.473009,"redis_calls":7,"redis_duration_s":0.0016179999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017235,"completed_at":"2026-07-01T18:04:33.255Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.515053] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.257Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4d67d508bad8fbbed89354b4","created_at":"2026-07-01T18:04:32.766Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.770Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4d67d508bad8fbbed89354b4: start","job_status":"start","queue_duration_s":0.486378,"scheduling_latency_s":0.486399} vm-test-run-nixbot-gitlab> gitlab # [ 205.516578] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.275Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4d67d508bad8fbbed89354b4","created_at":"2026-07-01T18:04:32.766Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.770Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4d67d508bad8fbbed89354b4: done: 0.018175 sec","job_status":"done","queue_duration_s":0.486378,"scheduling_latency_s":0.486399,"redis_calls":8,"redis_duration_s":0.001833,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018175,"completed_at":"2026-07-01T18:04:33.275Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.589321] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.276Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"92999df5c4e27ee868cc458c","created_at":"2026-07-01T18:04:32.772Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.774Z","job_size_bytes":66,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-92999df5c4e27ee868cc458c: start","job_status":"start","queue_duration_s":0.501873,"scheduling_latency_s":0.501895} vm-test-run-nixbot-gitlab> gitlab # [ 205.657292] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.291Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"92999df5c4e27ee868cc458c","created_at":"2026-07-01T18:04:32.772Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.774Z","job_size_bytes":66,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-92999df5c4e27ee868cc458c: done: 0.01482 sec","job_status":"done","queue_duration_s":0.501873,"scheduling_latency_s":0.501895,"redis_calls":7,"redis_duration_s":0.0015480000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01482,"completed_at":"2026-07-01T18:04:33.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.727121] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.292Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"84ce7ae56a7ba31c1eb0a68f","created_at":"2026-07-01T18:04:32.776Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.779Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-84ce7ae56a7ba31c1eb0a68f: start","job_status":"start","queue_duration_s":0.513714,"scheduling_latency_s":0.513735} vm-test-run-nixbot-gitlab> gitlab # [ 205.728393] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.314Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"84ce7ae56a7ba31c1eb0a68f","created_at":"2026-07-01T18:04:32.776Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.779Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-84ce7ae56a7ba31c1eb0a68f: done: 0.022184 sec","job_status":"done","queue_duration_s":0.513714,"scheduling_latency_s":0.513735,"redis_calls":7,"redis_duration_s":0.001567,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022184,"completed_at":"2026-07-01T18:04:33.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002365,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.803286] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.316Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"f2d7a1d292417d9246102adb","created_at":"2026-07-01T18:04:32.858Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.860Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f2d7a1d292417d9246102adb: start","job_status":"start","queue_duration_s":0.455411,"scheduling_latency_s":0.455433} vm-test-run-nixbot-gitlab> gitlab # [ 205.804370] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.373Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"f2d7a1d292417d9246102adb","created_at":"2026-07-01T18:04:32.858Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.860Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f2d7a1d292417d9246102adb: done: 0.05682 sec","job_status":"done","queue_duration_s":0.455411,"scheduling_latency_s":0.455433,"redis_calls":7,"redis_duration_s":0.001665,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05682,"completed_at":"2026-07-01T18:04:33.373Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.879827] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.374Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9259aa2f44592b59a7423990","created_at":"2026-07-01T18:04:32.861Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.864Z","job_size_bytes":6,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9259aa2f44592b59a7423990: start","job_status":"start","queue_duration_s":0.510231,"scheduling_latency_s":0.510253} vm-test-run-nixbot-gitlab> gitlab # [ 205.883216] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"9259aa2f44592b59a7423990","created_at":"2026-07-01T18:04:32.861Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.864Z","job_size_bytes":6,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-9259aa2f44592b59a7423990: done: 0.013468 sec","job_status":"done","queue_duration_s":0.510231,"scheduling_latency_s":0.510253,"redis_calls":7,"redis_duration_s":0.001656,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001206,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013468,"completed_at":"2026-07-01T18:04:33.387Z","load_balancing_strategy":"primary","db_duration_s":0.000422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 205.955195] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"006fd9b327935bbc35c0f2b8","created_at":"2026-07-01T18:04:32.865Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.871Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-006fd9b327935bbc35c0f2b8: start","job_status":"start","queue_duration_s":0.517488,"scheduling_latency_s":0.517508} vm-test-run-nixbot-gitlab> gitlab # [ 206.030912] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"006fd9b327935bbc35c0f2b8","created_at":"2026-07-01T18:04:32.865Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.871Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-006fd9b327935bbc35c0f2b8: done: 0.024515 sec","job_status":"done","queue_duration_s":0.517488,"scheduling_latency_s":0.517508,"redis_calls":10,"redis_duration_s":0.002168,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341943,"duration_s":0.024515,"completed_at":"2026-07-01T18:04:33.413Z","load_balancing_strategy":"primary","db_duration_s":0.004103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 206.041235] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.415Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"42058a9b2cb12f9ab6ef9793","created_at":"2026-07-01T18:04:32.924Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.926Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-42058a9b2cb12f9ab6ef9793: start","job_status":"start","queue_duration_s":0.488383,"scheduling_latency_s":0.488404} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 206.121055] gitaly[1379]: time="2026-07-01T18:04:33.472Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:05:03.370" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:33.470" grpc.time_ms=0.989 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.08 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 206.122351] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.527Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"42058a9b2cb12f9ab6ef9793","created_at":"2026-07-01T18:04:32.924Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-6da8a20a6d4d4b9a","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:32.926Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-42058a9b2cb12f9ab6ef9793: done: 0.112248 sec","job_status":"done","queue_duration_s":0.488383,"scheduling_latency_s":0.488404,"gitaly_calls":3,"gitaly_duration_s":0.007894,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0076749999999999995,"redis_read_bytes":98,"redis_write_bytes":6014,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001169,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001749,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000272,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001625,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000236,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00051,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074318,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.092719326,"duration_s":0.112248,"completed_at":"2026-07-01T18:04:33.527Z","load_balancing_strategy":"primary","db_duration_s":0.030767,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 206.200191] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.529Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d7b6bdbee772415e2dab5fbf","created_at":"2026-07-01T18:04:33.461Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-43a8e5f13f034728","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:33.463Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d7b6bdbee772415e2dab5fbf: start","job_status":"start","queue_duration_s":0.065654,"scheduling_latency_s":0.065675} vm-test-run-nixbot-gitlab> gitlab # [ 206.201793] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:33.552Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d7b6bdbee772415e2dab5fbf","created_at":"2026-07-01T18:04:33.461Z","trace_propagation_headers":{"sentry-trace":"b03afe16bb1146a68e27d3b199e19fb5-43a8e5f13f034728","baggage":"sentry-trace_id=b03afe16bb1146a68e27d3b199e19fb5,sentry-release=Unknown"},"correlation_id":"01KWFDN81YHWRXV16Z7NC54RAB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:33.463Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d7b6bdbee772415e2dab5fbf: done: 0.023267 sec","job_status":"done","queue_duration_s":0.065654,"scheduling_latency_s":0.065675,"redis_calls":7,"redis_duration_s":0.0015099999999999998,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023267,"completed_at":"2026-07-01T18:04:33.552Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 206.276355] gitaly[1379]: time="2026-07-01T18:04:33.492Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:03.391" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:33.492" grpc.time_ms=0.516 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 206.280554] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 48 vm-test-run-nixbot-gitlab> gitlab # [ 206.284799] gitaly[1379]: time="2026-07-01T18:04:33.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDN81YHWRXV16Z7NC54RAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:03.416" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:33.517" grpc.time_ms=0.806 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 206.305298] gitaly[1379]: time="2026-07-01T18:04:34.215Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNA06Y8HBRMWRTKS6A8VS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:04.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:34.214" grpc.time_ms=0.614 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 207.357168] gitaly[1379]: time="2026-07-01T18:04:35.358Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNB2E5S5A1XG1A5NVRHJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:05.255" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:35.356" grpc.time_ms=1.274 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 207.385478] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 106 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.18 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 208.519820] gitaly[1379]: time="2026-07-01T18:04:36.520Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNC7AKWY7TJTVP5H7XKD9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:06.417" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:36.518" grpc.time_ms=1.506 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 208.553488] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 96 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.16 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 209.095767] nix-daemon[3333]: accepted connection from pid 3567, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 209.120886] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:37.121Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eb8b196a23393b5d244ccf65","created_at":"2026-07-01T18:04:16.513Z","trace_propagation_headers":{"sentry-trace":"87e1881e899b47f3934889520510e38b-e4234dca067c4d2d","baggage":"sentry-trace_id=87e1881e899b47f3934889520510e38b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMR1KE21QC8NYCH7QPAH4","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4560fea90e1d2cc7efb0a1325f194036f23e3aeec9772c15c3b57311ae8e0e1e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:37.116Z","retry_count":0,"job_size_bytes":451,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eb8b196a23393b5d244ccf65: start","job_status":"start","queue_duration_s":0.00408,"scheduling_latency_s":0.00484} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 209.176828] sidekiq-cluster[2103]: {"severity":"WARN","time":"2026-07-01T18:04:37.178Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"eb8b196a23393b5d244ccf65","created_at":"2026-07-01T18:04:16.513Z","trace_propagation_headers":{"sentry-trace":"87e1881e899b47f3934889520510e38b-e4234dca067c4d2d","baggage":"sentry-trace_id=87e1881e899b47f3934889520510e38b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDMR1KE21QC8NYCH7QPAH4","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:4560fea90e1d2cc7efb0a1325f194036f23e3aeec9772c15c3b57311ae8e0e1e","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:37.116Z","retry_count":0,"job_size_bytes":451,"pid":2103,"sidekiq_tid":"m9zb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-eb8b196a23393b5d244ccf65: fail: 0.057183 sec","job_status":"fail","queue_duration_s":0.00408,"scheduling_latency_s":0.00484,"redis_calls":7,"redis_duration_s":0.0040160000000000005,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057183,"completed_at":"2026-07-01T18:04:37.177Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.008021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 209.808162] gitaly[1379]: time="2026-07-01T18:04:37.809Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNDC7RNME67NF485NYXYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:07.708" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:37.808" grpc.time_ms=0.6 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.831696] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 193 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.27 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 209.838391] gitaly[1379]: time="2026-07-01T18:04:37.840Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:07.739" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:37.839" grpc.time_ms=0.492 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.870385] gitaly[1379]: time="2026-07-01T18:04:37.872Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-01T18:04:47.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:37.865" grpc.time_ms=6.563 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 209.932707] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 772 vm-test-run-nixbot-gitlab> gitlab # [ 209.934960] nixbot[3045]: {"ts": "2026-07-01T18:04:37.935Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/6bc363e76e16113eddc459ed4c9872dfa5b0cac0 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 209.950515] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:37.951Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d3048712723930884cf31f7f","created_at":"2026-07-01T18:04:37.917Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-808a3c10028c4718","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:37.921Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d3048712723930884cf31f7f: start","job_status":"start","queue_duration_s":0.030449,"scheduling_latency_s":0.03051} vm-test-run-nixbot-gitlab> gitlab # [ 210.097961] gitaly[1379]: time="2026-07-01T18:04:38.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:38.099" grpc.time_ms=0.367 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.123626] nix-daemon[3333]: accepted connection from pid 3591, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 210.156389] gitaly[1379]: time="2026-07-01T18:04:38.158Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:38.157" grpc.time_ms=0.47 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.197973] gitaly[1379]: time="2026-07-01T18:04:38.200Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3900 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-01T18:04:38.199" grpc.time_ms=0.32 pid=1379 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.216914] gitaly[1379]: time="2026-07-01T18:04:38.218Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" operations="[repack_refs]" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.234154] gitaly[1379]: time="2026-07-01T18:04:38.219Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" pid=1379 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-07-01 18:04:17.469810667 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.237252] gitaly[1379]: time="2026-07-01T18:04:38.238Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" pid=1379 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.238245] gitaly[1379]: time="2026-07-01T18:04:38.239Z" level=info msg="optimized repository" clean_stale_data_time=17.90032ms clean_work_trees_time="94.426µs" commit_graph_time="8.101µs" correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.94µs" pid=1379 prune_time="8.381µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=18.253996ms vm-test-run-nixbot-gitlab> gitlab # [ 210.239980] gitaly[1379]: time="2026-07-01T18:04:38.239Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.244605] gitaly[1379]: time="2026-07-01T18:04:38.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDNDWXT6ZFFDYWQTH9BFGT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:38.235" grpc.time_ms=10.39 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.316228] gitaly[1379]: time="2026-07-01T18:04:38.251Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 diskcache=8da6f9d1-2d14-48d4-be6a-7fca384ad06b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.317506] gitaly[1379]: time="2026-07-01T18:04:38.251Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=110 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8617 command.oublock=0 command.real_time_ms=161 command.spawn_token_fork_ms=2 command.system_time_ms=74 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-01T18:05:08.063" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-01T18:04:38.064" grpc.time_ms=187.143 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.318633] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 74 vm-test-run-nixbot-gitlab> gitlab # [ 210.319853] gitlab-workhorse[1279]: time="2026-07-01T18:04:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 210.320992] gitlab-workhorse[1279]: gitlab 2001:db8:1::1 - - [2026/07/01:18:04:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 53 vm-test-run-nixbot-gitlab> gitlab # [ 210.321950] gitaly[1379]: time="2026-07-01T18:04:38.295Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-01T18:05:08.194" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:38.294" grpc.time_ms=0.824 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 210.322509] gitaly[1379]: time="2026-07-01T18:04:38.301Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNDZ9JACF2R3K56N0CT2R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:38.289" pid=1379 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.396971] gitaly[1379]: time="2026-07-01T18:04:38.301Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNDZ9JACF2R3K56N0CT2R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:38.289" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.400715] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.357Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d3048712723930884cf31f7f","created_at":"2026-07-01T18:04:37.917Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-808a3c10028c4718","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:37.921Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d3048712723930884cf31f7f: done: 0.405441 sec","job_status":"done","queue_duration_s":0.030449,"scheduling_latency_s":0.03051,"gitaly_calls":4,"gitaly_duration_s":0.200584,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.020312,"redis_read_bytes":173,"redis_write_bytes":20924,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.0022,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002401,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.00503,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13377,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005013,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000639,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000868,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124017,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.355030597,"duration_s":0.405441,"completed_at":"2026-07-01T18:04:38.357Z","load_balancing_strategy":"primary","db_duration_s":0.03862,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 210.481178] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2301bd3dc8ef860257c7208d","created_at":"2026-07-01T18:04:38.049Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.053Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2301bd3dc8ef860257c7208d: start","job_status":"start","queue_duration_s":0.304929,"scheduling_latency_s":0.304951} vm-test-run-nixbot-gitlab> gitlab # [ 210.482759] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.390Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"2301bd3dc8ef860257c7208d","created_at":"2026-07-01T18:04:38.049Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.053Z","job_size_bytes":23,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-2301bd3dc8ef860257c7208d: done: 0.031927 sec","job_status":"done","queue_duration_s":0.304929,"scheduling_latency_s":0.304951,"redis_calls":7,"redis_duration_s":0.001719,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031927,"completed_at":"2026-07-01T18:04:38.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005269,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 210.594857] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.391Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"801905dfb2da135de2df628d","created_at":"2026-07-01T18:04:38.256Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.258Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-801905dfb2da135de2df628d: start","job_status":"start","queue_duration_s":0.132894,"scheduling_latency_s":0.13292} vm-test-run-nixbot-gitlab> gitlab # [ 210.596212] gitaly[1379]: time="2026-07-01T18:04:38.301Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNDZ9JACF2R3K56N0CT2R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-01T18:04:38.289" grpc.time_ms=11.957 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 210.597088] nix-daemon[3333]: accepted connection from pid 3648, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 210.597953] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.415Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"801905dfb2da135de2df628d","created_at":"2026-07-01T18:04:38.256Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.258Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-801905dfb2da135de2df628d: done: 0.024077 sec","job_status":"done","queue_duration_s":0.132894,"scheduling_latency_s":0.13292,"redis_calls":7,"redis_duration_s":0.004206,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024077,"completed_at":"2026-07-01T18:04:38.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 210.669264] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.417Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4a6d418bacf8f2e36309b5d8","created_at":"2026-07-01T18:04:38.265Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.267Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4a6d418bacf8f2e36309b5d8: start","job_status":"start","queue_duration_s":0.150003,"scheduling_latency_s":0.150026} vm-test-run-nixbot-gitlab> gitlab # [ 210.738295] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.562Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4a6d418bacf8f2e36309b5d8","created_at":"2026-07-01T18:04:38.265Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.267Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4a6d418bacf8f2e36309b5d8: done: 0.145 sec","job_status":"done","queue_duration_s":0.150003,"scheduling_latency_s":0.150026,"redis_calls":7,"redis_duration_s":0.0024809999999999997,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.145,"completed_at":"2026-07-01T18:04:38.562Z","load_balancing_strategy":"primary","db_duration_s":0.00676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 210.740762] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.564Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"ef619c2c3fb375c923ab09e2","created_at":"2026-07-01T18:04:38.275Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.277Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ef619c2c3fb375c923ab09e2: start","job_status":"start","queue_duration_s":0.286602,"scheduling_latency_s":0.286628} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses vm-test-run-nixbot-gitlab> gitlab # [ 210.819261] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.586Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"ef619c2c3fb375c923ab09e2","created_at":"2026-07-01T18:04:38.275Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.277Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ef619c2c3fb375c923ab09e2: done: 0.022785 sec","job_status":"done","queue_duration_s":0.286602,"scheduling_latency_s":0.286628,"redis_calls":10,"redis_duration_s":0.003385,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000629,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001614,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022785,"completed_at":"2026-07-01T18:04:38.586Z","load_balancing_strategy":"primary","db_duration_s":0.002731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 210.892868] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.588Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8a83fe4590bd16917baf4095","created_at":"2026-07-01T18:04:38.278Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.279Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8a83fe4590bd16917baf4095: start","job_status":"start","queue_duration_s":0.308919,"scheduling_latency_s":0.308941} vm-test-run-nixbot-gitlab> gitlab # [ 210.894363] gitaly[1379]: time="2026-07-01T18:04:38.319Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWFDNCX9F1VZ573ZR0DRC0G7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-01T18:05:08.217" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-01T18:04:38.318" grpc.time_ms=0.633 pid=1379 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6bc363e76e16113eddc459ed4c9872dfa5b0cac0/statuses, in 0.17 seconds) vm-test-run-nixbot-gitlab> [{'id': 3, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-01T18:04:27.379Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:32.371Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '6bc363e76e16113eddc459ed4c9872dfa5b0cac0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-01T18:04:32.569Z', 'started_at': None, 'finished_at': '2026-07-01T18:04:37.909Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 11.73 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 250.52 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 210.895417] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8a83fe4590bd16917baf4095","created_at":"2026-07-01T18:04:38.278Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.279Z","job_size_bytes":3,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8a83fe4590bd16917baf4095: done: 0.018896 sec","job_status":"done","queue_duration_s":0.308919,"scheduling_latency_s":0.308941,"redis_calls":7,"redis_duration_s":0.001797,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000474,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018896,"completed_at":"2026-07-01T18:04:38.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 210.950454] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.608Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7db372f20575f9868724d102","created_at":"2026-07-01T18:04:38.279Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.282Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7db372f20575f9868724d102: start","job_status":"start","queue_duration_s":0.326539,"scheduling_latency_s":0.326567} vm-test-run-nixbot-gitlab> test script finished in 250.58s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13) vm-test-run-nixbot-gitlab> gitlab # [ 211.047840] sidekiq-cluster[2103]: {"severity":"INFO","time":"2026-07-01T18:04:38.628Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7db372f20575f9868724d102","created_at":"2026-07-01T18:04:38.279Z","trace_propagation_headers":{"sentry-trace":"f9cabf7784844331a3829fc55085e874-cc79b7664c544d8d","baggage":"sentry-trace_id=f9cabf7784844331a3829fc55085e874,sentry-release=Unknown"},"correlation_id":"01KWFDNCX9F1VZ573ZR0DRC0G7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-01T18:04:38.282Z","job_size_bytes":19,"pid":2103,"sidekiq_tid":"m7qf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorkerqemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.14 seconds) post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab