nixbot

1these 13 derivations will be built:2  /nix/store/cs750yzvkgh0qx8gpwb4m7ifivnbzlfc-python3.13-nixbot.drv3  /nix/store/1mycljba1qqn2vmg2pdv3vv4cqj98kx7-unit-nixbot.service.drv4  /nix/store/pbdcvhaidjp6szc6r8fhx4myc8rkrj27-system-units.drv5  /nix/store/1041zgvknqbl1lq5ap921kcsyhs50nzh-etc.drv6  /nix/store/41xckyz2i8figjcw4j1lqwwifg00808y-activate.drv7  /nix/store/9hspw6gycq1aknzfndgxl9m6r7alp8q7-nixos-system-gitlab-test.drv8  /nix/store/8vrj35i5hfdvb74kj90b09rrwqqxs7a9-closure-info.drv9  /nix/store/1hqc1fxaw4l56b35dbgchbwj50b16jpz-closure-info.drv10  /nix/store/xnp152wyi4a41kv48ma462xywmrcy19b-run-nixos-vm.drv11  /nix/store/bwafkiv0ayv6b0366dg3x9xr0dxqr7n3-nixos-vm.drv12  /nix/store/wdikv74pkqsn9iv7vsvw03b8jgi9bzyn-driverConfiguration.json.drv13  /nix/store/7i9r6g668g0izs3lz03krzl7virv0jn8-nixos-test-driver-nixbot-gitlab.drv14  /nix/store/3igbisc9abkdndmn4bpbdkgfkciwm2r3-vm-test-run-nixbot-gitlab.drv15building '/nix/store/cs750yzvkgh0qx8gpwb4m7ifivnbzlfc-python3.13-nixbot.drv' on 'ssh-ng://nix@eliza'16building '/nix/store/cs750yzvkgh0qx8gpwb4m7ifivnbzlfc-python3.13-nixbot.drv'17python3.13-nixbot> Sourcing python-remove-tests-dir-hook18python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh19python3.13-nixbot> Sourcing python-remove-bin-bytecode-hook.sh20python3.13-nixbot> Sourcing pypa-build-hook21python3.13-nixbot> Using pypaBuildPhase22python3.13-nixbot> Sourcing python-runtime-deps-check-hook23python3.13-nixbot> Using pythonRuntimeDepsCheckHook24python3.13-nixbot> Sourcing pypa-install-hook25python3.13-nixbot> Using pypaInstallPhase26python3.13-nixbot> Sourcing python-imports-check-hook.sh27python3.13-nixbot> Using pythonImportsCheckPhase28python3.13-nixbot> Sourcing python-namespaces-hook29python3.13-nixbot> Sourcing python-catch-conflicts-hook.sh30python3.13-nixbot> Running phase: unpackPhase31python3.13-nixbot> unpacking source archive /nix/store/210dgxy12c30dqqcc812v0f0c2imycdm-nixbot32python3.13-nixbot> source root is nixbot33python3.13-nixbot> setting SOURCE_DATE_EPOCH to timestamp 315619200 of file "nixbot/pyproject.toml"34python3.13-nixbot> Running phase: patchPhase35python3.13-nixbot> Running phase: updateAutotoolsGnuConfigScriptsPhase36python3.13-nixbot> Running phase: configurePhase37python3.13-nixbot> no configure script, doing nothing38python3.13-nixbot> Running phase: buildPhase39python3.13-nixbot> Executing pypaBuildPhase40python3.13-nixbot> Creating a wheel...41python3.13-nixbot> pypa build flags: --no-isolation --outdir dist/ --wheel42python3.13-nixbot> * Getting build dependencies for wheel...43python3.13-nixbot> * Building wheel...44python3.13-nixbot> Successfully built nixbot-0.0.1-py3-none-any.whl45python3.13-nixbot> Finished creating a wheel...46python3.13-nixbot> Finished executing pypaBuildPhase47python3.13-nixbot> Running phase: pythonRuntimeDepsCheckHook48python3.13-nixbot> Executing pythonRuntimeDepsCheck49python3.13-nixbot> Checking runtime dependencies for nixbot-0.0.1-py3-none-any.whl50python3.13-nixbot> Finished executing pythonRuntimeDepsCheck51python3.13-nixbot> Running phase: installPhase52python3.13-nixbot> Executing pypaInstallPhase53python3.13-nixbot> Successfully installed nixbot-0.0.1-py3-none-any.whl54python3.13-nixbot> Finished executing pypaInstallPhase55python3.13-nixbot> Running phase: pythonOutputDistPhase56python3.13-nixbot> Executing pythonOutputDistPhase57python3.13-nixbot> Finished executing pythonOutputDistPhase58python3.13-nixbot> Running phase: fixupPhase59python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot60python3.13-nixbot> checking for references to /build/ in /nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot...61python3.13-nixbot> patching script interpreter paths in /nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot62python3.13-nixbot> stripping (with command strip and flags -S -p) in  /nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot/lib /nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot/bin63python3.13-nixbot> shrinking RPATHs of ELF executables and libraries in /nix/store/hb426s6g73gwfd8nsqhl38rcip5q0vhg-python3.13-nixbot-dist64python3.13-nixbot> checking for references to /build/ in /nix/store/hb426s6g73gwfd8nsqhl38rcip5q0vhg-python3.13-nixbot-dist...65python3.13-nixbot> patching script interpreter paths in /nix/store/hb426s6g73gwfd8nsqhl38rcip5q0vhg-python3.13-nixbot-dist66python3.13-nixbot> Rewriting #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13 to #!/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.1367python3.13-nixbot> wrapping `/nix/store/ij00alw0892ppjs10qyra3vzczy7dbpn-python3.13-nixbot/bin/nixbot'...68python3.13-nixbot> Executing pythonRemoveTestsDir69python3.13-nixbot> Finished executing pythonRemoveTestsDir70python3.13-nixbot> Running phase: pythonCatchConflictsPhase71python3.13-nixbot> Running phase: pythonRemoveBinBytecodePhase72python3.13-nixbot> Running phase: pythonImportsCheckPhase73python3.13-nixbot> Executing pythonImportsCheckPhase74building '/nix/store/1mycljba1qqn2vmg2pdv3vv4cqj98kx7-unit-nixbot.service.drv'75unit-nixbot.service> tribuchet: building on eliza76building '/nix/store/pbdcvhaidjp6szc6r8fhx4myc8rkrj27-system-units.drv'77system-units> tribuchet: building on eliza78building '/nix/store/1041zgvknqbl1lq5ap921kcsyhs50nzh-etc.drv'79etc> tribuchet: building on eliza80building '/nix/store/41xckyz2i8figjcw4j1lqwwifg00808y-activate.drv'81activate> tribuchet: building on eliza82building '/nix/store/9hspw6gycq1aknzfndgxl9m6r7alp8q7-nixos-system-gitlab-test.drv'83nixos-system-gitlab-test> tribuchet: building on eliza84nixos-system-gitlab-test> structuredAttrs is enabled85building '/nix/store/8vrj35i5hfdvb74kj90b09rrwqqxs7a9-closure-info.drv'86closure-info> tribuchet: building on eliza87closure-info> structuredAttrs is enabled88building '/nix/store/1hqc1fxaw4l56b35dbgchbwj50b16jpz-closure-info.drv'89closure-info> tribuchet: building on eliza90closure-info> structuredAttrs is enabled91building '/nix/store/xnp152wyi4a41kv48ma462xywmrcy19b-run-nixos-vm.drv'92run-nixos-vm> tribuchet: building on eliza93building '/nix/store/bwafkiv0ayv6b0366dg3x9xr0dxqr7n3-nixos-vm.drv'94nixos-vm> tribuchet: building on eliza95building '/nix/store/wdikv74pkqsn9iv7vsvw03b8jgi9bzyn-driverConfiguration.json.drv'96driverConfiguration.json> tribuchet: building on eliza97driverConfiguration.json> structuredAttrs is enabled98building '/nix/store/7i9r6g668g0izs3lz03krzl7virv0jn8-nixos-test-driver-nixbot-gitlab.drv'99nixos-test-driver-nixbot-gitlab> tribuchet: building on eliza100nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck)101nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck102nixos-test-driver-nixbot-gitlab> All checks passed!103nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint)104nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint105nixos-test-driver-nixbot-gitlab> All checks passed!106building '/nix/store/3igbisc9abkdndmn4bpbdkgfkciwm2r3-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@eliza'107building '/nix/store/3igbisc9abkdndmn4bpbdkgfkciwm2r3-vm-test-run-nixbot-gitlab.drv'108vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state109vm-test-run-nixbot-gitlab> start all VLans110vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.00 seconds)111vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds112vm-test-run-nixbot-gitlab> run the VM test script113vm-test-run-nixbot-gitlab> additionally exposed symbols:114vm-test-run-nixbot-gitlab>     gitlab,115vm-test-run-nixbot-gitlab>     vlan1,116vm-test-run-nixbot-gitlab>     start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh117vm-test-run-nixbot-gitlab> start all VMs118vm-test-run-nixbot-gitlab> gitlab: starting vm119vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13)120vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image...121vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.lrgsfY7eAB', fmt=raw size=1073741824122vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025)123vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks:      0/262144             done124vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes125vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 76f7fb10-4600-4538-a0ba-fdc87be370c6126vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks:127vm-test-run-nixbot-gitlab> gitlab # 	32768, 98304, 163840, 229376128vm-test-run-nixbot-gitlab> gitlab # 129vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8   done130vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8   done131vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done132vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8   done133vm-test-run-nixbot-gitlab> gitlab # 134vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created.135vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image...136vm-test-run-nixbot-gitlab> gitlab # Created Nix store image.137vm-test-run-nixbot-gitlab> (finished: start all VMs, in 65.03 seconds)138vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service139vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting140vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]141vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Jun  1 15:51:08 UTC 2026142vm-test-run-nixbot-gitlab> gitlab # [    0.000000] KASLR enabled143vm-test-run-nixbot-gitlab> gitlab # [    0.000000] random: crng init done144vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Machine model: linux,dummy-virt145vm-test-run-nixbot-gitlab> gitlab # [    0.000000] efi: UEFI not found.146vm-test-run-nixbot-gitlab> gitlab # [    0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT147vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]148vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]149vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Zone ranges:150vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA      [mem 0x0000000040000000-0x00000000ffffffff]151vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   DMA32    empty152vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Normal   [mem 0x0000000100000000-0x00000001bfffffff]153vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   Device   empty154vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Movable zone start for each node155vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Early memory node ranges156vm-test-run-nixbot-gitlab> gitlab # [    0.000000]   node   0: [mem 0x0000000040000000-0x00000001bfffffff]157vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]158vm-test-run-nixbot-gitlab> gitlab # [    0.000000] cma: Reserved 32 MiB at 0x00000000fe000000159vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: probing for conduit method from DT.160vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: PSCIv1.3 detected in firmware.161vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Using standard PSCI v0.2 function IDs162vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: Trusted OS migration not required163vm-test-run-nixbot-gitlab> gitlab # [    0.000000] psci: SMC Calling Convention v1.1164vm-test-run-nixbot-gitlab> gitlab # [    0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)165vm-test-run-nixbot-gitlab> gitlab # [    0.000000] percpu: Embedded 76 pages/cpu s185944 r8192 d117160 u311296166vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Detected PIPT I-cache on CPU0167vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)168vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: GICv3 CPU interface169vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-v4170vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: Spectre-BHB171vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38172vm-test-run-nixbot-gitlab> gitlab # [    0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23173vm-test-run-nixbot-gitlab> gitlab # [    0.000000] alternatives: applying boot alternatives174vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/q5pdcrplk38k27np8170dgrngnnzvwgs-nixos-system-gitlab-test/init regInfo=/nix/store/dw977jixm7rcmfpg41llbbi4lag6apvg-closure-info/registration console=ttyAMA0,115200n8 console=tty0175vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Unknown kernel command line parameters "regInfo=/nix/store/dw977jixm7rcmfpg41llbbi4lag6apvg-closure-info/registration", will be passed to user space.176vm-test-run-nixbot-gitlab> gitlab # [    0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes177vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)178vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)179vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: area num 4.180vm-test-run-nixbot-gitlab> gitlab # [    0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)181vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Fallback order for Node 0: 0182vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1572864183vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Policy zone: Normal184vm-test-run-nixbot-gitlab> gitlab # [    0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off185vm-test-run-nixbot-gitlab> gitlab # [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1186vm-test-run-nixbot-gitlab> gitlab # [    0.000000] allocated 12582912 bytes of page_ext187vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocating 74648 entries in 292 pages188vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ftrace: allocated 292 pages with 3 groups189vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Hierarchical RCU implementation.190vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU event tracing is enabled.191vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.192vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Trampoline variant of Tasks RCU enabled.193vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Rude variant of Tasks RCU enabled.194vm-test-run-nixbot-gitlab> gitlab # [    0.000000] 	Tracing variant of Tasks RCU enabled.195vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.196vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4197vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.198vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.199vm-test-run-nixbot-gitlab> gitlab # [    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.200vm-test-run-nixbot-gitlab> gitlab # [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0201vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 256 SPIs implemented202vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: 0 Extended SPIs implemented203vm-test-run-nixbot-gitlab> gitlab # [    0.000000] Root IRQ handler: gic_handle_irq204vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI205vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0206vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000207vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS [mem 0x08080000-0x0809ffff]208vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)209vm-test-run-nixbot-gitlab> gitlab # [    0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)210vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: using LPI property table @0x0000000100f30000211vm-test-run-nixbot-gitlab> gitlab # [    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000212vm-test-run-nixbot-gitlab> gitlab # [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.213vm-test-run-nixbot-gitlab> gitlab # [    0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).214vm-test-run-nixbot-gitlab> gitlab # [    0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns215vm-test-run-nixbot-gitlab> gitlab # [    0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns216vm-test-run-nixbot-gitlab> gitlab # [    0.000027] arm-pv: using stolen time PV217vm-test-run-nixbot-gitlab> gitlab # [    0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)218vm-test-run-nixbot-gitlab> gitlab # [    0.000561] Console: colour dummy device 80x25219vm-test-run-nixbot-gitlab> gitlab # [    0.000568] printk: legacy console [tty0] enabled220vm-test-run-nixbot-gitlab> gitlab # [    0.000737] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)221vm-test-run-nixbot-gitlab> gitlab # [    0.000744] pid_max: default: 32768 minimum: 301222vm-test-run-nixbot-gitlab> gitlab # [    0.000833] LSM: initializing lsm=capability,landlock,yama,bpf,ima223vm-test-run-nixbot-gitlab> gitlab # [    0.000973] landlock: Up and running.224vm-test-run-nixbot-gitlab> gitlab # [    0.000976] Yama: becoming mindful.225vm-test-run-nixbot-gitlab> gitlab # [    0.001400] LSM support for eBPF active226vm-test-run-nixbot-gitlab> gitlab # [    0.001593] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)227vm-test-run-nixbot-gitlab> gitlab # [    0.001701] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)228vm-test-run-nixbot-gitlab> gitlab # [    0.002891] cacheinfo: Unable to detect cache hierarchy for CPU 0229vm-test-run-nixbot-gitlab> gitlab # [    0.003542] rcu: Hierarchical SRCU implementation.230vm-test-run-nixbot-gitlab> gitlab # [    0.003545] rcu: 	Max phase no-delay instances is 1000.231vm-test-run-nixbot-gitlab> gitlab # [    0.003703] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level232vm-test-run-nixbot-gitlab> gitlab # [    0.004739] fsl-mc MSI: its@8080000 domain created233vm-test-run-nixbot-gitlab> gitlab # [    0.004827] EFI services will not be available.234vm-test-run-nixbot-gitlab> gitlab # [    0.005011] smp: Bringing up secondary CPUs ...235vm-test-run-nixbot-gitlab> gitlab # [    0.005647] Detected PIPT I-cache on CPU1236vm-test-run-nixbot-gitlab> gitlab # [    0.005748] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000237vm-test-run-nixbot-gitlab> gitlab # [    0.005878] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000238vm-test-run-nixbot-gitlab> gitlab # [    0.006007] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]239vm-test-run-nixbot-gitlab> gitlab # [    0.007087] Detected PIPT I-cache on CPU2240vm-test-run-nixbot-gitlab> gitlab # [    0.007280] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000241vm-test-run-nixbot-gitlab> gitlab # [    0.007413] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000242vm-test-run-nixbot-gitlab> gitlab # [    0.007542] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]243vm-test-run-nixbot-gitlab> gitlab # [    0.008531] Detected PIPT I-cache on CPU3244vm-test-run-nixbot-gitlab> gitlab # [    0.008649] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000245vm-test-run-nixbot-gitlab> gitlab # [    0.008781] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000246vm-test-run-nixbot-gitlab> gitlab # [    0.008913] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]247vm-test-run-nixbot-gitlab> gitlab # [    0.009421] smp: Brought up 1 node, 4 CPUs248vm-test-run-nixbot-gitlab> gitlab # [    0.009447] SMP: Total of 4 processors activated.249vm-test-run-nixbot-gitlab> gitlab # [    0.009450] CPU: All CPU(s) started at EL1250vm-test-run-nixbot-gitlab> gitlab # [    0.009457] CPU features: detected: Branch Target Identification251vm-test-run-nixbot-gitlab> gitlab # [    0.009461] CPU features: detected: ARMv8.4 Translation Table Level252vm-test-run-nixbot-gitlab> gitlab # [    0.009464] CPU features: detected: Instruction cache invalidation not required for I/D coherence253vm-test-run-nixbot-gitlab> gitlab # [    0.009467] CPU features: detected: Data cache clean to the PoU not required for I/D coherence254vm-test-run-nixbot-gitlab> gitlab # [    0.009471] CPU features: detected: Common not Private translations255vm-test-run-nixbot-gitlab> gitlab # [    0.009473] CPU features: detected: CRC32 instructions256vm-test-run-nixbot-gitlab> gitlab # [    0.009476] CPU features: detected: Data cache clean to Point of Deep Persistence257vm-test-run-nixbot-gitlab> gitlab # [    0.009479] CPU features: detected: Data cache clean to Point of Persistence258vm-test-run-nixbot-gitlab> gitlab # [    0.009482] CPU features: detected: Data independent timing control (DIT)259vm-test-run-nixbot-gitlab> gitlab # [    0.009485] CPU features: detected: E0PD260vm-test-run-nixbot-gitlab> gitlab # [    0.009487] CPU features: detected: Enhanced Counter Virtualization261vm-test-run-nixbot-gitlab> gitlab # [    0.009490] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)262vm-test-run-nixbot-gitlab> gitlab # [    0.009493] CPU features: detected: Enhanced Virtualization Traps263vm-test-run-nixbot-gitlab> gitlab # [    0.009495] CPU features: detected: Fine Grained Traps264vm-test-run-nixbot-gitlab> gitlab # [    0.009498] CPU features: detected: Generic authentication (architected QARMA5 algorithm)265vm-test-run-nixbot-gitlab> gitlab # [    0.009501] CPU features: detected: RCpc load-acquire (LDAPR)266vm-test-run-nixbot-gitlab> gitlab # [    0.009504] CPU features: detected: LSE atomic instructions267vm-test-run-nixbot-gitlab> gitlab # [    0.009507] CPU features: detected: Privileged Access Never268vm-test-run-nixbot-gitlab> gitlab # [    0.009509] CPU features: detected: PMUv3269vm-test-run-nixbot-gitlab> gitlab # [    0.009512] CPU features: detected: RAS Extension Support270vm-test-run-nixbot-gitlab> gitlab # [    0.009514] CPU features: detected: RASv1p1 Extension Support271vm-test-run-nixbot-gitlab> gitlab # [    0.009517] CPU features: detected: Random Number Generator272vm-test-run-nixbot-gitlab> gitlab # [    0.009519] CPU features: detected: Speculation barrier (SB)273vm-test-run-nixbot-gitlab> gitlab # [    0.009521] CPU features: detected: Stage-2 Force Write-Back274vm-test-run-nixbot-gitlab> gitlab # [    0.009524] CPU features: detected: TLB range maintenance instructions275vm-test-run-nixbot-gitlab> gitlab # [    0.009528] CPU features: detected: Speculative Store Bypassing Safe (SSBS)276vm-test-run-nixbot-gitlab> gitlab # [    0.009661] alternatives: applying system-wide alternatives277vm-test-run-nixbot-gitlab> gitlab # [    0.012469] CPU features: detected: BBM Level 2 without TLB conflict abort278vm-test-run-nixbot-gitlab> gitlab # [    0.012664] Memory: 5965828K/6291456K available (24320K kernel code, 7086K rwdata, 26308K rodata, 4736K init, 1102K bss, 270904K reserved, 32768K cma-reserved)279vm-test-run-nixbot-gitlab> gitlab # [    0.014080] devtmpfs: initialized280vm-test-run-nixbot-gitlab> gitlab # [    0.018060] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns281vm-test-run-nixbot-gitlab> gitlab # [    0.018103] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)282vm-test-run-nixbot-gitlab> gitlab # [    0.018170] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).283vm-test-run-nixbot-gitlab> gitlab # [    0.018362] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL284vm-test-run-nixbot-gitlab> gitlab # [    0.018367] 0 pages in range for non-PLT usage285vm-test-run-nixbot-gitlab> gitlab # [    0.018368] 508336 pages in range for PLT usage286vm-test-run-nixbot-gitlab> gitlab # [    0.018488] pinctrl core: initialized pinctrl subsystem287vm-test-run-nixbot-gitlab> gitlab # [    0.019264] DMI not present or invalid.288vm-test-run-nixbot-gitlab> gitlab # [    0.022471] NET: Registered PF_NETLINK/PF_ROUTE protocol family289vm-test-run-nixbot-gitlab> gitlab # [    0.025107] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations290vm-test-run-nixbot-gitlab> gitlab # [    0.025368] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations291vm-test-run-nixbot-gitlab> gitlab # [    0.027215] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations292vm-test-run-nixbot-gitlab> gitlab # [    0.027235] audit: initializing netlink subsys (disabled)293vm-test-run-nixbot-gitlab> gitlab # [    0.027507] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1294vm-test-run-nixbot-gitlab> gitlab # [    0.027942] thermal_sys: Registered thermal governor 'fair_share'295vm-test-run-nixbot-gitlab> gitlab # [    0.027945] thermal_sys: Registered thermal governor 'bang_bang'296vm-test-run-nixbot-gitlab> gitlab # [    0.027951] thermal_sys: Registered thermal governor 'step_wise'297vm-test-run-nixbot-gitlab> gitlab # [    0.027954] thermal_sys: Registered thermal governor 'user_space'298vm-test-run-nixbot-gitlab> gitlab # [    0.027956] thermal_sys: Registered thermal governor 'power_allocator'299vm-test-run-nixbot-gitlab> gitlab # [    0.028033] cpuidle: using governor ladder300vm-test-run-nixbot-gitlab> gitlab # [    0.028095] cpuidle: using governor menu301vm-test-run-nixbot-gitlab> gitlab # [    0.028283] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.302vm-test-run-nixbot-gitlab> gitlab # [    0.028493] ASID allocator initialised with 65536 entries303vm-test-run-nixbot-gitlab> gitlab # [    0.029600] Serial: AMBA PL011 UART driver304vm-test-run-nixbot-gitlab> gitlab # [    0.034662] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1305vm-test-run-nixbot-gitlab> gitlab # [    0.034982] printk: console [ttyAMA0] enabled306vm-test-run-nixbot-gitlab> gitlab # [    0.046948] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages307vm-test-run-nixbot-gitlab> gitlab # [    0.046971] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page308vm-test-run-nixbot-gitlab> gitlab # [    0.046981] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages309vm-test-run-nixbot-gitlab> gitlab # [    0.046989] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page310vm-test-run-nixbot-gitlab> gitlab # [    0.046997] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages311vm-test-run-nixbot-gitlab> gitlab # [    0.047005] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page312vm-test-run-nixbot-gitlab> gitlab # [    0.047014] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages313vm-test-run-nixbot-gitlab> gitlab # [    0.047021] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page314vm-test-run-nixbot-gitlab> gitlab # [    0.059382] fbcon: Taking over console315vm-test-run-nixbot-gitlab> gitlab # [    0.059392] ACPI: Interpreter disabled.316vm-test-run-nixbot-gitlab> gitlab # [    0.060366] iommu: Default domain type: Translated317vm-test-run-nixbot-gitlab> gitlab # [    0.060383] iommu: DMA domain TLB invalidation policy: strict mode318vm-test-run-nixbot-gitlab> gitlab # [    0.070476] SCSI subsystem initialized319vm-test-run-nixbot-gitlab> gitlab # [    0.070817] usbcore: registered new interface driver usbfs320vm-test-run-nixbot-gitlab> gitlab # [    0.070850] usbcore: registered new interface driver hub321vm-test-run-nixbot-gitlab> gitlab # [    0.070867] usbcore: registered new device driver usb322vm-test-run-nixbot-gitlab> gitlab # [    0.071148] pps_core: LinuxPPS API ver. 1 registered323vm-test-run-nixbot-gitlab> gitlab # [    0.071152] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>324vm-test-run-nixbot-gitlab> gitlab # [    0.071161] PTP clock support registered325vm-test-run-nixbot-gitlab> gitlab # [    0.071221] EDAC MC: Ver: 3.0.0326vm-test-run-nixbot-gitlab> gitlab # [    0.071738] scmi_core: SCMI protocol bus registered327vm-test-run-nixbot-gitlab> gitlab # [    0.072644] FPGA manager framework328vm-test-run-nixbot-gitlab> gitlab # [    0.073283] vgaarb: loaded329vm-test-run-nixbot-gitlab> gitlab # [    0.075507] clocksource: Switched to clocksource arch_sys_counter330vm-test-run-nixbot-gitlab> gitlab # [    0.093107] VFS: Disk quotas dquot_6.6.0331vm-test-run-nixbot-gitlab> gitlab # [    0.093171] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)332vm-test-run-nixbot-gitlab> gitlab # [    0.093705] netfs: FS-Cache loaded333vm-test-run-nixbot-gitlab> gitlab # [    0.093834] pnp: PnP ACPI: disabled334vm-test-run-nixbot-gitlab> gitlab # [    0.100046] NET: Registered PF_INET protocol family335vm-test-run-nixbot-gitlab> gitlab # [    0.101092] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)336vm-test-run-nixbot-gitlab> gitlab # [    0.131720] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)337vm-test-run-nixbot-gitlab> gitlab # [    0.131766] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)338vm-test-run-nixbot-gitlab> gitlab # [    0.131814] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)339vm-test-run-nixbot-gitlab> gitlab # [    0.132074] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)340vm-test-run-nixbot-gitlab> gitlab # [    0.132654] TCP: Hash tables configured (established 65536 bind 65536)341vm-test-run-nixbot-gitlab> gitlab # [    0.132736] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)342vm-test-run-nixbot-gitlab> gitlab # [    0.132786] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)343vm-test-run-nixbot-gitlab> gitlab # [    0.132897] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)344vm-test-run-nixbot-gitlab> gitlab # [    0.133085] NET: Registered PF_UNIX/PF_LOCAL protocol family345vm-test-run-nixbot-gitlab> gitlab # [    0.133131] NET: Registered PF_XDP protocol family346vm-test-run-nixbot-gitlab> gitlab # [    0.133144] PCI: CLS 0 bytes, default 64347vm-test-run-nixbot-gitlab> gitlab # [    0.133531] Trying to unpack rootfs image as initramfs...348vm-test-run-nixbot-gitlab> gitlab # [    0.140050] kvm [1]: HYP mode not available349vm-test-run-nixbot-gitlab> gitlab # [    0.183680] Initialise system trusted keyrings350vm-test-run-nixbot-gitlab> gitlab # [    0.183977] workingset: timestamp_bits=42 max_order=21 bucket_order=0351vm-test-run-nixbot-gitlab> gitlab # [    0.184810] squashfs: version 4.0 (2009/01/31) Phillip Lougher352vm-test-run-nixbot-gitlab> gitlab # [    0.184930] 9p: Installing v9fs 9p2000 file system support353vm-test-run-nixbot-gitlab> gitlab # [    0.196830] Key type asymmetric registered354vm-test-run-nixbot-gitlab> gitlab # [    0.196834] Asymmetric key parser 'x509' registered355vm-test-run-nixbot-gitlab> gitlab # [    0.196873] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)356vm-test-run-nixbot-gitlab> gitlab # [    0.197064] io scheduler mq-deadline registered357vm-test-run-nixbot-gitlab> gitlab # [    0.197068] io scheduler kyber registered358vm-test-run-nixbot-gitlab> gitlab # [    0.203801] pl061_gpio 9030000.pl061: PL061 GPIO chip registered359vm-test-run-nixbot-gitlab> gitlab # [    0.208156] ledtrig-cpu: registered to indicate activity on CPUs360vm-test-run-nixbot-gitlab> gitlab # [    0.208481] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:361vm-test-run-nixbot-gitlab> gitlab # [    0.208492] pci-host-generic 4010000000.pcie:       IO 0x003eff0000..0x003effffff -> 0x0000000000362vm-test-run-nixbot-gitlab> gitlab # [    0.208501] pci-host-generic 4010000000.pcie:      MEM 0x0010000000..0x003efeffff -> 0x0010000000363vm-test-run-nixbot-gitlab> gitlab # [    0.208507] pci-host-generic 4010000000.pcie:      MEM 0x8000000000..0xffffffffff -> 0x8000000000364vm-test-run-nixbot-gitlab> gitlab # [    0.208525] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits365vm-test-run-nixbot-gitlab> gitlab # [    0.208544] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]366vm-test-run-nixbot-gitlab> gitlab # [    0.208619] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00367vm-test-run-nixbot-gitlab> gitlab # [    0.208626] pci_bus 0000:00: root bus resource [bus 00-ff]368vm-test-run-nixbot-gitlab> gitlab # [    0.208629] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]369vm-test-run-nixbot-gitlab> gitlab # [    0.208632] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]370vm-test-run-nixbot-gitlab> gitlab # [    0.208635] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]371vm-test-run-nixbot-gitlab> gitlab # [    0.208699] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint372vm-test-run-nixbot-gitlab> gitlab # [    0.209220] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint373vm-test-run-nixbot-gitlab> gitlab # [    0.209449] pci 0000:00:01.0: BAR 0 [io  0x0000-0x001f]374vm-test-run-nixbot-gitlab> gitlab # [    0.209461] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]375vm-test-run-nixbot-gitlab> gitlab # [    0.209489] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]376vm-test-run-nixbot-gitlab> gitlab # [    0.209501] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]377vm-test-run-nixbot-gitlab> gitlab # [    0.210070] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint378vm-test-run-nixbot-gitlab> gitlab # [    0.210292] pci 0000:00:02.0: BAR 0 [io  0x0000-0x001f]379vm-test-run-nixbot-gitlab> gitlab # [    0.210304] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]380vm-test-run-nixbot-gitlab> gitlab # [    0.210330] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]381vm-test-run-nixbot-gitlab> gitlab # [    0.210883] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint382vm-test-run-nixbot-gitlab> gitlab # [    0.211113] pci 0000:00:03.0: BAR 0 [io  0x0000-0x001f]383vm-test-run-nixbot-gitlab> gitlab # [    0.211126] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]384vm-test-run-nixbot-gitlab> gitlab # [    0.211153] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]385vm-test-run-nixbot-gitlab> gitlab # [    0.211730] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint386vm-test-run-nixbot-gitlab> gitlab # [    0.211951] pci 0000:00:04.0: BAR 0 [io  0x0000-0x001f]387vm-test-run-nixbot-gitlab> gitlab # [    0.211963] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]388vm-test-run-nixbot-gitlab> gitlab # [    0.211994] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]389vm-test-run-nixbot-gitlab> gitlab # [    0.212544] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint390vm-test-run-nixbot-gitlab> gitlab # [    0.212760] pci 0000:00:05.0: BAR 0 [io  0x0000-0x007f]391vm-test-run-nixbot-gitlab> gitlab # [    0.212772] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]392vm-test-run-nixbot-gitlab> gitlab # [    0.212799] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]393vm-test-run-nixbot-gitlab> gitlab # [    0.213351] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint394vm-test-run-nixbot-gitlab> gitlab # [    0.213571] pci 0000:00:06.0: BAR 0 [io  0x0000-0x007f]395vm-test-run-nixbot-gitlab> gitlab # [    0.213583] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]396vm-test-run-nixbot-gitlab> gitlab # [    0.213611] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]397vm-test-run-nixbot-gitlab> gitlab # [    0.214207] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint398vm-test-run-nixbot-gitlab> gitlab # [    0.214426] pci 0000:00:07.0: BAR 0 [io  0x0000-0x001f]399vm-test-run-nixbot-gitlab> gitlab # [    0.214438] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]400vm-test-run-nixbot-gitlab> gitlab # [    0.214468] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]401vm-test-run-nixbot-gitlab> gitlab # [    0.214485] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]402vm-test-run-nixbot-gitlab> gitlab # [    0.215037] pci 0000:00:08.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint403vm-test-run-nixbot-gitlab> gitlab # [    0.215264] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]404vm-test-run-nixbot-gitlab> gitlab # [    0.215292] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]405vm-test-run-nixbot-gitlab> gitlab # [    0.215843] pci 0000:00:09.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint406vm-test-run-nixbot-gitlab> gitlab # [    0.216072] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]407vm-test-run-nixbot-gitlab> gitlab # [    0.216104] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]408vm-test-run-nixbot-gitlab> gitlab # [    0.216579] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint409vm-test-run-nixbot-gitlab> gitlab # [    0.216806] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]410vm-test-run-nixbot-gitlab> gitlab # [    0.217097] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint411vm-test-run-nixbot-gitlab> gitlab # [    0.217417] pci 0000:00:0b.0: BAR 0 [io  0x0000-0x003f]412vm-test-run-nixbot-gitlab> gitlab # [    0.217429] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]413vm-test-run-nixbot-gitlab> gitlab # [    0.217455] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]414vm-test-run-nixbot-gitlab> gitlab # [    0.218013] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint415vm-test-run-nixbot-gitlab> gitlab # [    0.218233] pci 0000:00:0c.0: BAR 0 [io  0x0000-0x001f]416vm-test-run-nixbot-gitlab> gitlab # [    0.218244] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]417vm-test-run-nixbot-gitlab> gitlab # [    0.218274] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]418vm-test-run-nixbot-gitlab> gitlab # [    0.218949] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned419vm-test-run-nixbot-gitlab> gitlab # [    0.218955] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned420vm-test-run-nixbot-gitlab> gitlab # [    0.218959] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned421vm-test-run-nixbot-gitlab> gitlab # [    0.219006] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned422vm-test-run-nixbot-gitlab> gitlab # [    0.219059] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned423vm-test-run-nixbot-gitlab> gitlab # [    0.219106] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned424vm-test-run-nixbot-gitlab> gitlab # [    0.219159] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned425vm-test-run-nixbot-gitlab> gitlab # [    0.219210] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned426vm-test-run-nixbot-gitlab> gitlab # [    0.219258] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned427vm-test-run-nixbot-gitlab> gitlab # [    0.219311] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned428vm-test-run-nixbot-gitlab> gitlab # [    0.219362] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned429vm-test-run-nixbot-gitlab> gitlab # [    0.219406] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned430vm-test-run-nixbot-gitlab> gitlab # [    0.219481] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned431vm-test-run-nixbot-gitlab> gitlab # [    0.219543] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned432vm-test-run-nixbot-gitlab> gitlab # [    0.219560] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned433vm-test-run-nixbot-gitlab> gitlab # [    0.219578] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned434vm-test-run-nixbot-gitlab> gitlab # [    0.219595] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned435vm-test-run-nixbot-gitlab> gitlab # [    0.219613] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned436vm-test-run-nixbot-gitlab> gitlab # [    0.219631] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned437vm-test-run-nixbot-gitlab> gitlab # [    0.219649] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned438vm-test-run-nixbot-gitlab> gitlab # [    0.219667] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned439vm-test-run-nixbot-gitlab> gitlab # [    0.219685] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned440vm-test-run-nixbot-gitlab> gitlab # [    0.219703] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned441vm-test-run-nixbot-gitlab> gitlab # [    0.219722] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned442vm-test-run-nixbot-gitlab> gitlab # [    0.219740] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned443vm-test-run-nixbot-gitlab> gitlab # [    0.219758] pci 0000:00:05.0: BAR 0 [io  0x1000-0x107f]: assigned444vm-test-run-nixbot-gitlab> gitlab # [    0.219775] pci 0000:00:06.0: BAR 0 [io  0x1080-0x10ff]: assigned445vm-test-run-nixbot-gitlab> gitlab # [    0.219792] pci 0000:00:0b.0: BAR 0 [io  0x1100-0x113f]: assigned446vm-test-run-nixbot-gitlab> gitlab # [    0.219809] pci 0000:00:01.0: BAR 0 [io  0x1140-0x115f]: assigned447vm-test-run-nixbot-gitlab> gitlab # [    0.219827] pci 0000:00:02.0: BAR 0 [io  0x1160-0x117f]: assigned448vm-test-run-nixbot-gitlab> gitlab # [    0.219844] pci 0000:00:03.0: BAR 0 [io  0x1180-0x119f]: assigned449vm-test-run-nixbot-gitlab> gitlab # [    0.219861] pci 0000:00:04.0: BAR 0 [io  0x11a0-0x11bf]: assigned450vm-test-run-nixbot-gitlab> gitlab # [    0.219879] pci 0000:00:07.0: BAR 0 [io  0x11c0-0x11df]: assigned451vm-test-run-nixbot-gitlab> gitlab # [    0.219896] pci 0000:00:0c.0: BAR 0 [io  0x11e0-0x11ff]: assigned452vm-test-run-nixbot-gitlab> gitlab # [    0.219919] pci_bus 0000:00: resource 4 [io  0x0000-0xffff]453vm-test-run-nixbot-gitlab> gitlab # [    0.219924] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]454vm-test-run-nixbot-gitlab> gitlab # [    0.219927] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]455vm-test-run-nixbot-gitlab> gitlab # [    0.221396] pci 0000:00:0a.0: enabling device (0000 -> 0002)456vm-test-run-nixbot-gitlab> gitlab # [    0.233085] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)457vm-test-run-nixbot-gitlab> gitlab # [    0.235279] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)458vm-test-run-nixbot-gitlab> gitlab # [    0.237638] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)459vm-test-run-nixbot-gitlab> gitlab # [    0.239739] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)460vm-test-run-nixbot-gitlab> gitlab # [    0.241905] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)461vm-test-run-nixbot-gitlab> gitlab # [    0.245433] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)462vm-test-run-nixbot-gitlab> gitlab # [    0.249928] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)463vm-test-run-nixbot-gitlab> gitlab # [    0.252256] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)464vm-test-run-nixbot-gitlab> gitlab # [    0.254069] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)465vm-test-run-nixbot-gitlab> gitlab # [    0.256294] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)466vm-test-run-nixbot-gitlab> gitlab # [    0.260245] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)467vm-test-run-nixbot-gitlab> gitlab # [    0.268591] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled468vm-test-run-nixbot-gitlab> gitlab # [    0.270253] msm_serial: driver initialized469vm-test-run-nixbot-gitlab> gitlab # [    0.270373] SuperH (H)SCI(F) driver initialized470vm-test-run-nixbot-gitlab> gitlab # [    0.270421] STM32 USART driver initialized471vm-test-run-nixbot-gitlab> gitlab # [    0.288547] loop: module loaded472vm-test-run-nixbot-gitlab> gitlab # [    0.288736] virtio_blk virtio4: 4/0/0 default/read/poll queues473vm-test-run-nixbot-gitlab> gitlab # [    0.290728] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)474vm-test-run-nixbot-gitlab> gitlab # [    0.292589] virtio_blk virtio5: 4/0/0 default/read/poll queues475vm-test-run-nixbot-gitlab> gitlab # [    0.294368] virtio_blk virtio5: [vdb] 11827248 512-byte logical blocks (6.06 GB/5.64 GiB)476vm-test-run-nixbot-gitlab> gitlab # [    0.296517] megasas: 07.734.00.00-rc1477vm-test-run-nixbot-gitlab> gitlab # [    0.297116] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]478vm-test-run-nixbot-gitlab> gitlab # [    0.299885] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000479vm-test-run-nixbot-gitlab> gitlab # [    0.299898] Intel/Sharp Extended Query Table at 0x0031480vm-test-run-nixbot-gitlab> gitlab # [    0.301551] Using buffer write method481vm-test-run-nixbot-gitlab> gitlab # [    0.301578] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]482vm-test-run-nixbot-gitlab> gitlab # [    0.303301] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000483vm-test-run-nixbot-gitlab> gitlab # [    0.303314] Intel/Sharp Extended Query Table at 0x0031484vm-test-run-nixbot-gitlab> gitlab # [    0.304947] Using buffer write method485vm-test-run-nixbot-gitlab> gitlab # [    0.304958] Concatenating MTD devices:486vm-test-run-nixbot-gitlab> gitlab # [    0.304961] (0): "0.flash"487vm-test-run-nixbot-gitlab> gitlab # [    0.304963] (1): "0.flash"488vm-test-run-nixbot-gitlab> gitlab # [    0.304965] into device "0.flash"489vm-test-run-nixbot-gitlab> gitlab # [    0.630717] Freeing initrd memory: 25372K490vm-test-run-nixbot-gitlab> gitlab # [    0.647995] tun: Universal TUN/TAP device driver, 1.6491vm-test-run-nixbot-gitlab> gitlab # [    0.656571] thunder_xcv, ver 1.0492vm-test-run-nixbot-gitlab> gitlab # [    0.656690] thunder_bgx, ver 1.0493vm-test-run-nixbot-gitlab> gitlab # [    0.656759] nicpf, ver 1.0494vm-test-run-nixbot-gitlab> gitlab # [    0.658396] e1000: Intel(R) PRO/1000 Network Driver495vm-test-run-nixbot-gitlab> gitlab # [    0.658408] e1000: Copyright (c) 1999-2006 Intel Corporation.496vm-test-run-nixbot-gitlab> gitlab # [    0.658598] e1000e: Intel(R) PRO/1000 Network Driver497vm-test-run-nixbot-gitlab> gitlab # [    0.658620] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.498vm-test-run-nixbot-gitlab> gitlab # [    0.658716] igb: Intel(R) Gigabit Ethernet Network Driver499vm-test-run-nixbot-gitlab> gitlab # [    0.658727] igb: Copyright (c) 2007-2014 Intel Corporation.500vm-test-run-nixbot-gitlab> gitlab # [    0.658792] igbvf: Intel(R) Gigabit Virtual Function Network Driver501vm-test-run-nixbot-gitlab> gitlab # [    0.658803] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.502vm-test-run-nixbot-gitlab> gitlab # [    0.659216] sky2: driver version 1.30503vm-test-run-nixbot-gitlab> gitlab # [    0.664225] usbcore: registered new interface driver usb-storage504vm-test-run-nixbot-gitlab> gitlab # [    0.664693] ehci-pci 0000:00:0a.0: EHCI Host Controller505vm-test-run-nixbot-gitlab> gitlab # [    0.664706] usbcore: registered new interface driver usbserial_generic506vm-test-run-nixbot-gitlab> gitlab # [    0.664754] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1507vm-test-run-nixbot-gitlab> gitlab # [    0.664762] usbserial: USB Serial support registered for generic508vm-test-run-nixbot-gitlab> gitlab # [    0.665524] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000509vm-test-run-nixbot-gitlab> gitlab # [    0.666606] hv_vmbus: registering driver hyperv_keyboard510vm-test-run-nixbot-gitlab> gitlab # [    0.671652] rtc-pl031 9010000.pl031: registered as rtc0511vm-test-run-nixbot-gitlab> gitlab # [    0.671728] rtc-pl031 9010000.pl031: setting system clock to 2026-07-04T10:49:47 UTC (1783162187)512vm-test-run-nixbot-gitlab> gitlab # [    0.672670] i2c_dev: i2c /dev entries driver513vm-test-run-nixbot-gitlab> gitlab # [    0.675680] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00514vm-test-run-nixbot-gitlab> gitlab # [    0.676971] hub 1-0:1.0: USB hub found515vm-test-run-nixbot-gitlab> gitlab # [    0.677082] hub 1-0:1.0: 6 ports detected516vm-test-run-nixbot-gitlab> gitlab # [    0.681027] sdhci: Secure Digital Host Controller Interface driver517vm-test-run-nixbot-gitlab> gitlab # [    0.681041] sdhci: Copyright(c) Pierre Ossman518vm-test-run-nixbot-gitlab> gitlab # [    0.681846] Synopsys Designware Multimedia Card Interface Driver519vm-test-run-nixbot-gitlab> gitlab # [    0.683022] sdhci-pltfm: SDHCI platform and OF driver helper520vm-test-run-nixbot-gitlab> gitlab # [    0.687854] hid: raw HID events driver (C) Jiri Kosina521vm-test-run-nixbot-gitlab> gitlab # [    0.688754] usbcore: registered new interface driver usbhid522vm-test-run-nixbot-gitlab> gitlab # [    0.688765] usbhid: USB HID core driver523vm-test-run-nixbot-gitlab> gitlab # [    0.723313] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available524vm-test-run-nixbot-gitlab> gitlab # [    0.727681] drop_monitor: Initializing network drop monitor service525vm-test-run-nixbot-gitlab> gitlab # [    0.728029] NET: Registered PF_INET6 protocol family526vm-test-run-nixbot-gitlab> gitlab # [    0.729741] Segment Routing with IPv6527vm-test-run-nixbot-gitlab> gitlab # [    0.729773] In-situ OAM (IOAM) with IPv6528vm-test-run-nixbot-gitlab> gitlab # [    0.729833] NET: Registered PF_PACKET protocol family529vm-test-run-nixbot-gitlab> gitlab # [    0.730133] 9pnet: Installing 9P2000 support530vm-test-run-nixbot-gitlab> gitlab # [    0.734195] Key type dns_resolver registered531vm-test-run-nixbot-gitlab> gitlab # [    0.751876] registered taskstats version 1532vm-test-run-nixbot-gitlab> gitlab # [    0.752496] Loading compiled-in X.509 certificates533vm-test-run-nixbot-gitlab> gitlab # [    0.775042] Demotion targets for Node 0: null534vm-test-run-nixbot-gitlab> gitlab # [    0.775260] Key type .fscrypt registered535vm-test-run-nixbot-gitlab> gitlab # [    0.775267] Key type fscrypt-provisioning registered536vm-test-run-nixbot-gitlab> gitlab # [    0.775452] ima: No TPM chip found, activating TPM-bypass!537vm-test-run-nixbot-gitlab> gitlab # [    0.775475] ima: Allocated hash algorithm: sha1538vm-test-run-nixbot-gitlab> gitlab # [    0.775531] ima: No architecture policies found539vm-test-run-nixbot-gitlab> gitlab # [    0.776591] input: gpio-keys as /devices/platform/gpio-keys/input/input0540vm-test-run-nixbot-gitlab> gitlab # [    0.803472] clk: Disabling unused clocks541vm-test-run-nixbot-gitlab> gitlab # [    0.803542] PM: genpd: Disabling unused power domains542vm-test-run-nixbot-gitlab> gitlab # [    0.809501] Freeing unused kernel memory: 4736K543vm-test-run-nixbot-gitlab> gitlab # [    0.809750] Run /init as init process544vm-test-run-nixbot-gitlab> gitlab # [    0.842035] systemd[1]: Successfully made /usr/ read-only.545vm-test-run-nixbot-gitlab> gitlab # [    0.923597] usb 1-1: new high-speed USB device number 2 using ehci-pci546vm-test-run-nixbot-gitlab> gitlab # [    1.082511] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1547vm-test-run-nixbot-gitlab> gitlab # [    1.172275] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0548vm-test-run-nixbot-gitlab> gitlab # [    1.176579] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)549vm-test-run-nixbot-gitlab> gitlab # [    1.176620] systemd[1]: Detected virtualization qemu.550vm-test-run-nixbot-gitlab> gitlab # [    1.176678] systemd[1]: Detected architecture arm64.551vm-test-run-nixbot-gitlab> gitlab # [    1.176692] systemd[1]: Running in initrd.552vm-test-run-nixbot-gitlab> gitlab # [    1.177644] systemd[1]: Initializing machine ID from random generator.553vm-test-run-nixbot-gitlab> gitlab # [    1.177886] systemd[1]: Hostname set to <gitlab>.554vm-test-run-nixbot-gitlab> gitlab # [    1.295588] usb 1-2: new high-speed USB device number 3 using ehci-pci555vm-test-run-nixbot-gitlab> gitlab # [    1.339397] systemd[1]: Queued start job for default target Initrd Default Target.556vm-test-run-nixbot-gitlab> gitlab # [    1.454124] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2557vm-test-run-nixbot-gitlab> gitlab # [    1.454618] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0558vm-test-run-nixbot-gitlab> gitlab # [    1.580358] systemd[1]: Created slice Slice /system/modprobe.559vm-test-run-nixbot-gitlab> gitlab # [    1.580906] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.560vm-test-run-nixbot-gitlab> gitlab # [    1.580975] systemd[1]: Expecting device /dev/disk/by-label/nix-store...561vm-test-run-nixbot-gitlab> gitlab # [    1.581005] systemd[1]: Expecting device /dev/disk/by-label/nixos...562vm-test-run-nixbot-gitlab> gitlab # [    1.581052] systemd[1]: Reached target Path Units.563vm-test-run-nixbot-gitlab> gitlab # [    1.581088] systemd[1]: Reached target Slice Units.564vm-test-run-nixbot-gitlab> gitlab # [    1.581129] systemd[1]: Reached target Swaps.565vm-test-run-nixbot-gitlab> gitlab # [    1.581165] systemd[1]: Reached target Timer Units.566vm-test-run-nixbot-gitlab> gitlab # [    1.581480] systemd[1]: Listening on D-Bus System Message Bus Socket.567vm-test-run-nixbot-gitlab> gitlab # [    1.581797] systemd[1]: Listening on Journal Socket (/dev/log).568vm-test-run-nixbot-gitlab> gitlab # [    1.582113] systemd[1]: Listening on Journal Sockets.569vm-test-run-nixbot-gitlab> gitlab # [    1.582398] systemd[1]: Listening on udev Control Socket.570vm-test-run-nixbot-gitlab> gitlab # [    1.582598] systemd[1]: Listening on udev Kernel Socket.571vm-test-run-nixbot-gitlab> gitlab # [    1.582635] systemd[1]: Reached target Socket Units.572vm-test-run-nixbot-gitlab> gitlab # [    1.589786] systemd[1]: Starting Create List of Static Device Nodes...573vm-test-run-nixbot-gitlab> gitlab # [    1.597035] systemd[1]: Starting Load Kernel Module 9pnet_virtio...574vm-test-run-nixbot-gitlab> gitlab # [    1.597171] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs575vm-test-run-nixbot-gitlab> gitlab # [    1.605192] systemd[1]: Mounting Kernel Configuration File System...576vm-test-run-nixbot-gitlab> gitlab # [    1.628426] systemd[1]: Starting Journal Service...577vm-test-run-nixbot-gitlab> gitlab # [    1.635688] systemd[1]: Starting Load Kernel Modules...578vm-test-run-nixbot-gitlab> gitlab # [    1.635815] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki579vm-test-run-nixbot-gitlab> gitlab # [    1.642219] systemd[1]: Starting Coldplug All udev Devices...580vm-test-run-nixbot-gitlab> gitlab # [    1.671465] systemd[1]: Finished Create List of Static Device Nodes.581vm-test-run-nixbot-gitlab> gitlab # [    1.674048] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.582vm-test-run-nixbot-gitlab> gitlab # [    1.684119] systemd[1]: Finished Load Kernel Module 9pnet_virtio.583vm-test-run-nixbot-gitlab> gitlab # [    1.698702] systemd-journald[155]: Collecting audit messages is disabled.584vm-test-run-nixbot-gitlab> gitlab # [    1.699051] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.585vm-test-run-nixbot-gitlab> gitlab # [    1.699923] systemd[1]: Mounted Kernel Configuration File System.586vm-test-run-nixbot-gitlab> gitlab # [    1.704626] [drm] pci: virtio-gpu-pci detected at 0000:00:09.0587vm-test-run-nixbot-gitlab> gitlab # [    1.705113] [drm] features: -virgl +edid -resource_blob -host_visible588vm-test-run-nixbot-gitlab> gitlab # [    1.705120] [drm] features: -context_init589vm-test-run-nixbot-gitlab> gitlab # [    1.707693] [drm] number of scanouts: 1590vm-test-run-nixbot-gitlab> gitlab # [    1.707722] [drm] number of cap sets: 0591vm-test-run-nixbot-gitlab> gitlab # [    1.707724] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev592vm-test-run-nixbot-gitlab> gitlab # [    1.708517] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...593vm-test-run-nixbot-gitlab> gitlab # [    1.709439] virtio-pci 0000:00:09.0: [drm] Registered 1 planes with drm panic594vm-test-run-nixbot-gitlab> gitlab # [    1.709451] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:09.0 on minor 0595vm-test-run-nixbot-gitlab> gitlab # [    1.736702] Console: switching to colour frame buffer device 160x50596vm-test-run-nixbot-gitlab> gitlab # [    1.746495] virtio-pci 0000:00:09.0: [drm] fb0: virtio_gpudrmfb frame buffer device597vm-test-run-nixbot-gitlab> gitlab # [    1.752377] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.598vm-test-run-nixbot-gitlab> gitlab # [    1.763221] systemd[1]: Starting Create Static Device Nodes in /dev...599vm-test-run-nixbot-gitlab> gitlab # [    1.768259] systemd[1]: Finished Load Kernel Modules.600vm-test-run-nixbot-gitlab> gitlab # [    1.769878] systemd-modules-load[157]: Using 4 probe threads601vm-test-run-nixbot-gitlab> gitlab # [    1.771220] systemd-modules-load[157]: Module 'virtio_balloon' is built in602vm-test-run-nixbot-gitlab> gitlab # [    1.772546] systemd-modules-load[157]: Module 'virtio_console' is built in603vm-test-run-nixbot-gitlab> gitlab # [    1.773754] systemd-modules-load[157]: Module 'virtio_rng' is built in604vm-test-run-nixbot-gitlab> gitlab # [    1.774888] systemd-modules-load[157]: Inserted module 'dm_mod'605vm-test-run-nixbot-gitlab> gitlab # [    1.775963] systemd-modules-load[157]: Inserted module 'virtio_gpu'[    1.779929] systemd[1]: Starting Apply Kernel Variables...606vm-test-run-nixbot-gitlab> gitlab # 607vm-test-run-nixbot-gitlab> gitlab # [    1.782656] systemd[1]: Started Journal Service.608vm-test-run-nixbot-gitlab> gitlab # [    1.809035] systemd[1]: Finished Create Static Device Nodes in /dev.609vm-test-run-nixbot-gitlab> gitlab # [    1.813063] systemd[1]: Finished Apply Kernel Variables.610vm-test-run-nixbot-gitlab> gitlab # [    1.814415] systemd[1]: Reached target Preparation for Local File Systems.611vm-test-run-nixbot-gitlab> gitlab # [    1.815916] systemd[1]: Reached target Local File Systems.612vm-test-run-nixbot-gitlab> gitlab # [    1.824431] systemd[1]: Starting Create System Files and Directories...613vm-test-run-nixbot-gitlab> gitlab # [    1.841873] systemd[1]: Starting Rule-based Manager for Device Events and Files...614vm-test-run-nixbot-gitlab> gitlab # [    1.857937] systemd[1]: Finished Create System Files and Directories.615vm-test-run-nixbot-gitlab> gitlab # [    1.888316] systemd-udevd[196]: Using default interface naming scheme 'v260'.616vm-test-run-nixbot-gitlab> gitlab # [    1.912777] systemd[1]: Started Rule-based Manager for Device Events and Files.617vm-test-run-nixbot-gitlab> gitlab # [    1.942839] systemd[1]: Starting Virtual Console Setup...618vm-test-run-nixbot-gitlab> gitlab # [    2.000325] systemd-vconsole-setup[212]: Configuration of first virtual console was skipped, ignoring remaining ones.619vm-test-run-nixbot-gitlab> gitlab # [    2.005003] systemd[1]: Finished Virtual Console Setup.620vm-test-run-nixbot-gitlab> gitlab # [    2.406792] systemd[1]: Finished Coldplug All udev Devices.621vm-test-run-nixbot-gitlab> gitlab # [    2.409460] systemd[1]: Reached target System Initialization.622vm-test-run-nixbot-gitlab> gitlab # [    2.412039] systemd[1]: Reached target Basic System.623vm-test-run-nixbot-gitlab> gitlab # [    2.584557] (udev-worker)[200]: Network interface NamePolicy= disabled on kernel command line.624vm-test-run-nixbot-gitlab> gitlab # [    2.619752] systemd[1]: Found device /dev/disk/by-label/nixos.625vm-test-run-nixbot-gitlab> gitlab # [    2.621131] systemd[1]: Reached target Initrd Root Device.626vm-test-run-nixbot-gitlab> gitlab # [    2.622686] (udev-worker)[200]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.627vm-test-run-nixbot-gitlab> gitlab # [    2.626446] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...628vm-test-run-nixbot-gitlab> gitlab # [    2.643032] systemd[1]: Found device /dev/disk/by-label/nix-store.629vm-test-run-nixbot-gitlab> gitlab # [    2.680680] systemd-fsck[237]: nixos: clean, 12/65536 files, 13019/262144 blocks630vm-test-run-nixbot-gitlab> gitlab # [    2.688652] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.631vm-test-run-nixbot-gitlab> gitlab # [    2.693124] systemd[1]: Mounting /sysroot...632vm-test-run-nixbot-gitlab> gitlab # [    2.729081] EXT4-fs (vda): mounted filesystem 76f7fb10-4600-4538-a0ba-fdc87be370c6 r/w with ordered data mode. Quota mode: none.633vm-test-run-nixbot-gitlab> gitlab # [    2.730595] systemd[1]: Mounted /sysroot.634vm-test-run-nixbot-gitlab> gitlab # [    2.731425] systemd[1]: Reached target Initrd Root File System.635vm-test-run-nixbot-gitlab> gitlab # [    2.736646] systemd[1]: Mounting /sysroot/nix/.ro-store...636vm-test-run-nixbot-gitlab> gitlab # [    2.744585] systemd[1]: Mounting /sysroot/nix/.rw-store...637vm-test-run-nixbot-gitlab> gitlab # [    2.759957] systemd[1]: Mounting /sysroot/run...638vm-test-run-nixbot-gitlab> gitlab # [    2.768515] systemd[1]: Mounting /sysroot/tmp/shared...639vm-test-run-nixbot-gitlab> gitlab # [    2.776388] systemd[1]: Mounting /sysroot/tmp/xchg...640vm-test-run-nixbot-gitlab> gitlab # [    2.785124] erofs (device vdb): mounted with root inode @ nid 128.641vm-test-run-nixbot-gitlab> gitlab # [    2.787646] systemd[1]: Starting Mountpoints Configured in the Real Root...642vm-test-run-nixbot-gitlab> gitlab # [    2.800950] systemd-sysroot-fstab-check[259]: /sysroot should be mounted in the initrd, will request daemon-reload.643vm-test-run-nixbot-gitlab> gitlab # [    2.817963] systemd[1]: Mounted /sysroot/nix/.ro-store.644vm-test-run-nixbot-gitlab> gitlab # [    2.838603] systemd[1]: Mounted /sysroot/nix/.rw-store.645vm-test-run-nixbot-gitlab> gitlab # [    2.842698] systemd[1]: Mounted /sysroot/run.646vm-test-run-nixbot-gitlab> gitlab # [    2.844928] systemd[1]: Mounted /sysroot/tmp/shared.647vm-test-run-nixbot-gitlab> gitlab # [    2.867286] systemd[1]: Starting rw-sysroot-nix-store.service...648vm-test-run-nixbot-gitlab> gitlab # [    2.873138] systemd[1]: Reload requested from client PID 259 ('systemd-sysroot') (unit initrd-parse-etc.service)...649vm-test-run-nixbot-gitlab> gitlab # [    2.875040] systemd[1]: Reloading...650vm-test-run-nixbot-gitlab> gitlab # [    3.157395] systemd[1]: Reloading finished in 283 ms.651vm-test-run-nixbot-gitlab> gitlab # [    3.197670] systemd-sysroot-fstab-check[259]: Requesting initrd-fs.target/start/replace...652vm-test-run-nixbot-gitlab> gitlab # [    3.262374] (udev-worker)[231]: mtd0ro: Failed to find and pin callout binary "/nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/mtd_probe": No such file or directory653vm-test-run-nixbot-gitlab> gitlab # [    3.265374] (udev-worker)[231]: mtd0ro: /nix/store/3x4plcsrj2r4mmzis46wrmyka8g4jhi8-systemd-260.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory654vm-test-run-nixbot-gitlab> gitlab # [    3.293642] systemd[1]: Mounted /sysroot/tmp/xchg.655vm-test-run-nixbot-gitlab> gitlab # [    3.294606] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.656vm-test-run-nixbot-gitlab> gitlab # [    3.296246] systemd[1]: Finished rw-sysroot-nix-store.service.657vm-test-run-nixbot-gitlab> gitlab # [    3.303975] systemd-sysroot-fstab-check[259]: Requesting swap.target/start/replace...658vm-test-run-nixbot-gitlab> gitlab # [    3.308578] systemd[1]: initrd-parse-etc.service: Deactivated successfully.659vm-test-run-nixbot-gitlab> gitlab # [    3.310457] systemd[1]: Finished Mountpoints Configured in the Real Root.660vm-test-run-nixbot-gitlab> gitlab # [    3.311659] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.661vm-test-run-nixbot-gitlab> gitlab # [    3.321649] systemd[1]: Starting rw-sysroot-nix-store.service...662vm-test-run-nixbot-gitlab> gitlab # [    3.322749] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.663vm-test-run-nixbot-gitlab> gitlab # [    3.324002] systemd[1]: Stopped Virtual Console Setup.664vm-test-run-nixbot-gitlab> gitlab # [    3.327824] systemd[1]: Stopping Virtual Console Setup...665vm-test-run-nixbot-gitlab> gitlab # [    3.340257] systemd[1]: Starting Virtual Console Setup...666vm-test-run-nixbot-gitlab> gitlab # [    3.342441] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.667vm-test-run-nixbot-gitlab> gitlab # [    3.345288] systemd[1]: Finished rw-sysroot-nix-store.service.668vm-test-run-nixbot-gitlab> gitlab # [    3.368480] systemd-vconsole-setup[489]: Configuration of first virtual console was skipped, ignoring remaining ones.669vm-test-run-nixbot-gitlab> gitlab # [    3.372788] systemd[1]: Finished Virtual Console Setup.670vm-test-run-nixbot-gitlab> gitlab # [    3.672333] systemd[1]: Mounting /sysroot/nix/store...671vm-test-run-nixbot-gitlab> gitlab # [    3.747353] systemd[1]: Mounted /sysroot/nix/store.672vm-test-run-nixbot-gitlab> gitlab # [    3.749791] systemd[1]: Reached target Initrd File Systems.673vm-test-run-nixbot-gitlab> gitlab # [    3.756287] systemd[1]: Starting Find NixOS closure...674vm-test-run-nixbot-gitlab> gitlab # [    3.763485] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...675vm-test-run-nixbot-gitlab> gitlab # [    3.809127] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.676vm-test-run-nixbot-gitlab> gitlab # [    3.812812] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.677vm-test-run-nixbot-gitlab> gitlab # [    3.835071] systemd[1]: Finished Find NixOS closure.678vm-test-run-nixbot-gitlab> gitlab # [    3.837593] systemd[1]: Reached target Initrd Default Target.679vm-test-run-nixbot-gitlab> gitlab # [    3.844113] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...680vm-test-run-nixbot-gitlab> gitlab # [    3.894137] systemd[1]: Stopped target Initrd Default Target.681vm-test-run-nixbot-gitlab> gitlab # [    3.896918] systemd[1]: Stopped target Basic System.682vm-test-run-nixbot-gitlab> gitlab # [    3.899175] systemd[1]: Stopped target Initrd Root Device.683vm-test-run-nixbot-gitlab> gitlab # [    3.901715] systemd[1]: Stopped target Path Units.684vm-test-run-nixbot-gitlab> gitlab # [    3.903915] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.685vm-test-run-nixbot-gitlab> gitlab # [    3.907366] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.686vm-test-run-nixbot-gitlab> gitlab # [    3.910864] systemd[1]: Stopped target Slice Units.687vm-test-run-nixbot-gitlab> gitlab # [    3.913163] systemd[1]: Stopped target Socket Units.688vm-test-run-nixbot-gitlab> gitlab # [    3.915401] systemd[1]: Stopped target System Initialization.689vm-test-run-nixbot-gitlab> gitlab # [    3.918050] systemd[1]: Stopped target Swaps.690vm-test-run-nixbot-gitlab> gitlab # [    3.920137] systemd[1]: Stopped target Timer Units.691vm-test-run-nixbot-gitlab> gitlab # [    3.922368] systemd[1]: dbus.socket: Deactivated successfully.692vm-test-run-nixbot-gitlab> gitlab # [    3.925036] systemd[1]: Closed D-Bus System Message Bus Socket.693vm-test-run-nixbot-gitlab> gitlab # [    3.927681] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.694vm-test-run-nixbot-gitlab> gitlab # [    3.931149] systemd[1]: Stopped Find NixOS closure.695vm-test-run-nixbot-gitlab> gitlab # [    3.933279] systemd[1]: Starting Load Kernel Module 9pnet_virtio...696vm-test-run-nixbot-gitlab> gitlab # [    3.935748] systemd[1]: Starting rw-sysroot-nix-store.service...697vm-test-run-nixbot-gitlab> gitlab # [    3.938091] systemd[1]: systemd-sysctl.service: Deactivated successfully.698vm-test-run-nixbot-gitlab> gitlab # [    3.940477] systemd[1]: Stopped Apply Kernel Variables.699vm-test-run-nixbot-gitlab> gitlab # [    3.942295] systemd[1]: systemd-modules-load.service: Deactivated successfully.700vm-test-run-nixbot-gitlab> gitlab # [    3.944648] systemd[1]: Stopped Load Kernel Modules.701vm-test-run-nixbot-gitlab> gitlab # [    3.949962] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.702vm-test-run-nixbot-gitlab> gitlab # [    3.953865] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.703vm-test-run-nixbot-gitlab> gitlab # [    3.959911] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.704vm-test-run-nixbot-gitlab> gitlab # [    3.963174] systemd[1]: Stopped Create System Files and Directories.705vm-test-run-nixbot-gitlab> gitlab # [    3.969277] systemd[1]: Stopped target Local File Systems.706vm-test-run-nixbot-gitlab> gitlab # [    3.971718] systemd[1]: Stopped target Preparation for Local File Systems.707vm-test-run-nixbot-gitlab> gitlab # [    3.974809] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.708vm-test-run-nixbot-gitlab> gitlab # [    3.978041] systemd[1]: Stopped Coldplug All udev Devices.709vm-test-run-nixbot-gitlab> gitlab # [    3.980529] systemd[1]: Stopping Rule-based Manager for Device Events and Files...710vm-test-run-nixbot-gitlab> gitlab # [    3.983754] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.711vm-test-run-nixbot-gitlab> gitlab # [    3.988339] systemd[1]: Stopped Virtual Console Setup.712vm-test-run-nixbot-gitlab> gitlab # [    3.999167] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.713vm-test-run-nixbot-gitlab> gitlab # [    4.004601] systemd[1]: Finished Load Kernel Module 9pnet_virtio.714vm-test-run-nixbot-gitlab> gitlab # [    4.014519] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.715vm-test-run-nixbot-gitlab> gitlab # [    4.019636] systemd[1]: Finished rw-sysroot-nix-store.service.716vm-test-run-nixbot-gitlab> gitlab # [    4.029340] systemd[1]: systemd-udevd.service: Deactivated successfully.717vm-test-run-nixbot-gitlab> gitlab # [    4.034291] systemd[1]: Stopped Rule-based Manager for Device Events and Files.718vm-test-run-nixbot-gitlab> gitlab # [    4.037557] systemd[1]: systemd-udevd.service: Consumed 2.398s CPU time over 2.188s wall clock time, 32.5M memory peak.719vm-test-run-nixbot-gitlab> gitlab # [    4.053656] systemd[1]: initrd-cleanup.service: Deactivated successfully.720vm-test-run-nixbot-gitlab> gitlab # [    4.058674] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.721vm-test-run-nixbot-gitlab> gitlab # [    4.072782] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.722vm-test-run-nixbot-gitlab> gitlab # [    4.075970] systemd[1]: Closed udev Control Socket.723vm-test-run-nixbot-gitlab> gitlab # [    4.087982] systemd[1]: Starting Cleanup udev Database...724vm-test-run-nixbot-gitlab> gitlab # [    4.090454] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.725vm-test-run-nixbot-gitlab> gitlab # [    4.093939] systemd[1]: Stopped Create Static Device Nodes in /dev.726vm-test-run-nixbot-gitlab> gitlab # [    4.097363] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.727vm-test-run-nixbot-gitlab> gitlab # [    4.100980] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.728vm-test-run-nixbot-gitlab> gitlab # [    4.105625] systemd[1]: kmod-static-nodes.service: Deactivated successfully.729vm-test-run-nixbot-gitlab> gitlab # [    4.108726] systemd[1]: Stopped Create List of Static Device Nodes.730vm-test-run-nixbot-gitlab> gitlab # [    4.189603] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.731vm-test-run-nixbot-gitlab> gitlab # [    4.193103] systemd[1]: Finished Cleanup udev Database.732vm-test-run-nixbot-gitlab> gitlab # [    4.200864] systemd[1]: Reached target Switch Root.733vm-test-run-nixbot-gitlab> gitlab # [    4.206787] systemd[1]: Starting NixOS Activation...734vm-test-run-nixbot-gitlab> gitlab # [    4.424434] initrd-nixos-activation-start[575]: booting system configuration /nix/store/q5pdcrplk38k27np8170dgrngnnzvwgs-nixos-system-gitlab-test735vm-test-run-nixbot-gitlab> gitlab # [    4.495079] initrd-nixos-activation-start[575]: running activation script...736vm-test-run-nixbot-gitlab> gitlab # [    4.838384] initrd-nixos-activation-start[598]: setting up /etc...737vm-test-run-nixbot-gitlab> gitlab # [    5.019778] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.738vm-test-run-nixbot-gitlab> gitlab # [    5.024157] systemd[1]: Finished NixOS Activation.739vm-test-run-nixbot-gitlab> gitlab # [    5.042679] systemd[1]: Starting Switch Root...740vm-test-run-nixbot-gitlab> gitlab # [    5.093800] systemd[1]: Switching root.741vm-test-run-nixbot-gitlab> gitlab # [    5.676474] systemd-journald[155]: Received SIGTERM from PID 1 (systemd).742vm-test-run-nixbot-gitlab> gitlab # [    6.371480] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)743vm-test-run-nixbot-gitlab> gitlab # [    6.371648] systemd[1]: Detected virtualization qemu.744vm-test-run-nixbot-gitlab> gitlab # [    6.371722] systemd[1]: Detected architecture arm64.745vm-test-run-nixbot-gitlab> gitlab # [    6.371870] systemd[1]: Detected first boot.746vm-test-run-nixbot-gitlab> gitlab # [    6.373417] systemd[1]: Initializing machine ID from random generator.747vm-test-run-nixbot-gitlab> gitlab # [    6.602128] systemd[1]: bpf-restrict-fs: LSM BPF program attached748vm-test-run-nixbot-gitlab> gitlab # [    6.698969] systemd[1]: Applying preset policy.749vm-test-run-nixbot-gitlab> gitlab # [    6.915101] systemd[1]: Populated /etc with preset unit settings.750vm-test-run-nixbot-gitlab> gitlab # [    7.087061] systemd[1]: initrd-switch-root.service: Deactivated successfully.751vm-test-run-nixbot-gitlab> gitlab # [    7.088107] systemd[1]: Stopped initrd-switch-root.service.752vm-test-run-nixbot-gitlab> gitlab # [    7.094305] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.753vm-test-run-nixbot-gitlab> gitlab # [    7.099761] systemd[1]: Created slice Slice /system/getty.754vm-test-run-nixbot-gitlab> gitlab # [    7.103475] systemd[1]: Created slice GitLab DevOps Platform Slice.755vm-test-run-nixbot-gitlab> gitlab # [    7.107560] systemd[1]: Created slice User and Session Slice.756vm-test-run-nixbot-gitlab> gitlab # [    7.108985] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.757vm-test-run-nixbot-gitlab> gitlab # [    7.110309] systemd[1]: Started Forward Password Requests to Wall Directory Watch.758vm-test-run-nixbot-gitlab> gitlab # [    7.111474] systemd[1]: Expecting device /dev/hvc0...759vm-test-run-nixbot-gitlab> gitlab # [    7.112579] systemd[1]: Expecting device /dev/ttyAMA0...760vm-test-run-nixbot-gitlab> gitlab # [    7.113643] systemd[1]: Reached target Local Encrypted Volumes.761vm-test-run-nixbot-gitlab> gitlab # [    7.114772] systemd[1]: Stopped target initrd-fs.target.762vm-test-run-nixbot-gitlab> gitlab # [    7.115925] systemd[1]: Stopped target initrd-root-fs.target.763vm-test-run-nixbot-gitlab> gitlab # [    7.116984] systemd[1]: Stopped target initrd-switch-root.target.764vm-test-run-nixbot-gitlab> gitlab # [    7.118072] systemd[1]: Reached target Virtual Machines and Containers.765vm-test-run-nixbot-gitlab> gitlab # [    7.119219] systemd[1]: Reached target Path Units.766vm-test-run-nixbot-gitlab> gitlab # [    7.120354] systemd[1]: Reached target Remote File Systems.767vm-test-run-nixbot-gitlab> gitlab # [    7.121403] systemd[1]: Reached target Slice Units.768vm-test-run-nixbot-gitlab> gitlab # [    7.122477] systemd[1]: Reached target Swaps.769vm-test-run-nixbot-gitlab> gitlab # [    7.126879] systemd[1]: Listening on Process Core Dump Socket.770vm-test-run-nixbot-gitlab> gitlab # [    7.130198] systemd[1]: Listening on Credential Encryption/Decryption.771vm-test-run-nixbot-gitlab> gitlab # [    7.139630] systemd[1]: Starting Journal Log Access Socket...772vm-test-run-nixbot-gitlab> gitlab # [    7.141660] systemd[1]: Listening on Journal Audit Socket.773vm-test-run-nixbot-gitlab> gitlab # [    7.143183] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.774vm-test-run-nixbot-gitlab> gitlab # [    7.144207] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki775vm-test-run-nixbot-gitlab> gitlab # [    7.145426] systemd[1]: Listening on udev Control Socket.776vm-test-run-nixbot-gitlab> gitlab # [    7.161990] systemd[1]: Mounting Huge Pages File System...777vm-test-run-nixbot-gitlab> gitlab # [    7.173075] systemd[1]: Mounting POSIX Message Queue File System...778vm-test-run-nixbot-gitlab> gitlab # [    7.181028] systemd[1]: Mounting Kernel Debug File System...779vm-test-run-nixbot-gitlab> gitlab # [    7.186817] systemd[1]: Mounting Kernel Trace File System...780vm-test-run-nixbot-gitlab> gitlab # [    7.195748] systemd[1]: Starting Create List of Static Device Nodes...781vm-test-run-nixbot-gitlab> gitlab # [    7.207814] systemd[1]: Starting Load Kernel Module 9pnet_virtio...782vm-test-run-nixbot-gitlab> gitlab # [    7.209095] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs783vm-test-run-nixbot-gitlab> gitlab # [    7.223786] systemd[1]: Mounting Kernel Configuration File System...784vm-test-run-nixbot-gitlab> gitlab # [    7.231091] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm785vm-test-run-nixbot-gitlab> gitlab # [    7.235640] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore786vm-test-run-nixbot-gitlab> gitlab # [    7.240805] systemd[1]: Starting Load Kernel Module fuse...787vm-test-run-nixbot-gitlab> gitlab # [    7.241818] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67788vm-test-run-nixbot-gitlab> gitlab # [    7.249367] systemd[1]: Starting Journal Service...789vm-test-run-nixbot-gitlab> gitlab # [    7.254019] systemd[1]: Starting Load Kernel Modules...790vm-test-run-nixbot-gitlab> gitlab # [    7.263735] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...791vm-test-run-nixbot-gitlab> gitlab # [    7.275919] systemd[1]: Starting Remount Root and Kernel File Systems...792vm-test-run-nixbot-gitlab> gitlab # [    7.277166] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki793vm-test-run-nixbot-gitlab> gitlab # [    7.292924] fuse: init (API version 7.45)794vm-test-run-nixbot-gitlab> gitlab # [    7.295913] systemd[1]: Starting Coldplug All udev Devices...795vm-test-run-nixbot-gitlab> gitlab # [    7.305940] systemd[1]: Listening on Journal Log Access Socket.796vm-test-run-nixbot-gitlab> gitlab # [    7.307972] systemd-journald[807]: Collecting audit messages is enabled.797vm-test-run-nixbot-gitlab> gitlab # [    7.308564] systemd[1]: Mounted Huge Pages File System.798vm-test-run-nixbot-gitlab> gitlab # [    7.312058] systemd[1]: Mounted POSIX Message Queue File System.799vm-test-run-nixbot-gitlab> gitlab # [    7.315379] systemd[1]: Mounted Kernel Debug File System.800vm-test-run-nixbot-gitlab> gitlab # [    7.313960] systemd[1]: Queued start job for default target Multi-User System.801vm-test-run-nixbot-gitlab> gitlab # [    7.315905] systemd[1]: Unnecessary job was removed for /dev/vdb.[    7.319592] systemd[1]: Started Journal Service.802vm-test-run-nixbot-gitlab> gitlab # 803vm-test-run-nixbot-gitlab> gitlab # [    7.319254] systemd[1]: systemd-journald.service: Deactivated successfully.804vm-test-run-nixbot-gitlab> gitlab # [    7.321258] systemd-modules-load[808]: Using 3 probe threads805vm-test-run-nixbot-gitlab> gitlab # [    7.322672] systemd-modules-load[808]: Module 'atkbd' is built in806vm-test-run-nixbot-gitlab> gitlab # [    7.324140] systemd-modules-load[808]: Module 'loop' is built in807vm-test-run-nixbot-gitlab> gitlab # [    7.325541] systemd-modules-load[808]: Inserted module 'tls'808vm-test-run-nixbot-gitlab> gitlab # [    7.326894] systemd[1]: Mounted Kernel Trace File System.809vm-test-run-nixbot-gitlab> gitlab # [    7.328179] systemd[1]: Finished Create List of Static Device Nodes.810vm-test-run-nixbot-gitlab> gitlab # [    7.329640] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.811vm-test-run-nixbot-gitlab> gitlab # [    7.331783] systemd[1]: Finished Load Kernel Module 9pnet_virtio.812vm-test-run-nixbot-gitlab> gitlab # [    7.333291] systemd[1]: Mounted Kernel Configuration File System.813vm-test-run-nixbot-gitlab> gitlab # [    7.334601] systemd[1]: modprobe@fuse.service: Deactivated successfully.814vm-test-run-nixbot-gitlab> gitlab # [    7.335967] systemd[1]: Finished Load Kernel Module fuse.815vm-test-run-nixbot-gitlab> gitlab # [    7.337127] systemd[1]: Finished Load Kernel Modules.[    7.341155] EXT4-fs (vda): re-mounted 76f7fb10-4600-4538-a0ba-fdc87be370c6.816vm-test-run-nixbot-gitlab> gitlab # 817vm-test-run-nixbot-gitlab> gitlab # [    7.341387] systemd[1]: Mounting FUSE Control File System...818vm-test-run-nixbot-gitlab> gitlab # [    7.344258] systemd-oomd[809]: No swap; memory pressure usage will be degraded819vm-test-run-nixbot-gitlab> gitlab # [    7.349034] systemd[1]: Starting Firewall...820vm-test-run-nixbot-gitlab> gitlab # [    7.373726] systemd[1]: Starting Apply Kernel Variables...821vm-test-run-nixbot-gitlab> gitlab # [    7.380607] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...822vm-test-run-nixbot-gitlab> gitlab # [    7.387530] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.823vm-test-run-nixbot-gitlab> gitlab # [    7.393827] systemd[1]: Finished Remount Root and Kernel File Systems.824vm-test-run-nixbot-gitlab> gitlab # [    7.397283] systemd[1]: Mounted FUSE Control File System.825vm-test-run-nixbot-gitlab> gitlab # [    7.407304] systemd[1]: Starting Flush Journal to Persistent Storage...826vm-test-run-nixbot-gitlab> gitlab # [    7.411263] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore827vm-test-run-nixbot-gitlab> gitlab # [    7.415461] systemd[1]: Starting Load/Save OS Random Seed...828vm-test-run-nixbot-gitlab> gitlab # [    7.420179] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki829vm-test-run-nixbot-gitlab> gitlab # [    7.421708] systemd[1]: Finished Apply Kernel Variables.830vm-test-run-nixbot-gitlab> gitlab # [    7.428774] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.831vm-test-run-nixbot-gitlab> gitlab # [    7.436236] systemd[1]: Starting Create Static Device Nodes in /dev...832vm-test-run-nixbot-gitlab> gitlab # [    7.447395] systemd-journald[807]: Received client request to flush runtime journal.833vm-test-run-nixbot-gitlab> gitlab # [    7.478747] systemd[1]: Finished Load/Save OS Random Seed.834vm-test-run-nixbot-gitlab> gitlab # [    7.480130] systemd[1]: Reached target First Boot Complete.835vm-test-run-nixbot-gitlab> gitlab # [    7.481281] systemd[1]: Finished Create Static Device Nodes in /dev.836vm-test-run-nixbot-gitlab> gitlab # [    7.482480] systemd[1]: Reached target Preparation for Local File Systems.837vm-test-run-nixbot-gitlab> gitlab # [    7.483722] systemd[1]: Starting Rule-based Manager for Device Events and Files...838vm-test-run-nixbot-gitlab> gitlab # [    7.488555] systemd[1]: Finished Flush Journal to Persistent Storage.839vm-test-run-nixbot-gitlab> gitlab # [    7.537340] systemd-udevd[857]: Using default interface naming scheme 'v260'.840vm-test-run-nixbot-gitlab> gitlab # [    7.591041] systemd[1]: Started Rule-based Manager for Device Events and Files.841vm-test-run-nixbot-gitlab> gitlab # [    7.766647] systemd[1]: Finished Firewall.842vm-test-run-nixbot-gitlab> gitlab # [    8.070795] systemd[1]: Finished Coldplug All udev Devices.843vm-test-run-nixbot-gitlab> gitlab # [    8.087623] systemd[1]: Mounting /run/wrappers...844vm-test-run-nixbot-gitlab> gitlab # [    8.116950] systemd[1]: Mounted /run/wrappers.845vm-test-run-nixbot-gitlab> gitlab # [    8.119324] systemd[1]: Reached target Local File Systems.846vm-test-run-nixbot-gitlab> gitlab # [    8.121904] systemd[1]: Listening on Boot Loader Control Service Socket.847vm-test-run-nixbot-gitlab> gitlab # [    8.126332] systemd[1]: Starting register-nix-paths.service...848vm-test-run-nixbot-gitlab> gitlab # [    8.130634] systemd[1]: Starting Create SUID/SGID Wrappers...849vm-test-run-nixbot-gitlab> gitlab # [    8.134153] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.850vm-test-run-nixbot-gitlab> gitlab # [    8.144293] systemd[1]: Starting Save Transient machine-id to Disk...851vm-test-run-nixbot-gitlab> gitlab # [    8.149110] systemd[1]: Starting Create System Files and Directories...852vm-test-run-nixbot-gitlab> gitlab # [    8.188537] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs853vm-test-run-nixbot-gitlab> gitlab # [    8.210714] systemd[1]: Finished Create System Files and Directories.854vm-test-run-nixbot-gitlab> gitlab # [    8.215741] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.855vm-test-run-nixbot-gitlab> gitlab # [    8.219699] systemd[1]: Finished Save Transient machine-id to Disk.856vm-test-run-nixbot-gitlab> gitlab # [    8.226106] systemd[1]: Starting Rebuild Journal Catalog...857vm-test-run-nixbot-gitlab> gitlab # [    8.231370] systemd[1]: Starting Record System Boot/Shutdown in UTMP...858vm-test-run-nixbot-gitlab> gitlab # [    8.281007] systemd[1]: Finished Record System Boot/Shutdown in UTMP.859vm-test-run-nixbot-gitlab> gitlab # [    8.287362] (udev-worker)[914]: Network interface NamePolicy= disabled on kernel command line.860vm-test-run-nixbot-gitlab> gitlab # [    8.306568] systemd[1]: Finished Rebuild Journal Catalog.861vm-test-run-nixbot-gitlab> gitlab # [    8.318597] systemd[1]: Starting Update is Completed...862vm-test-run-nixbot-gitlab> gitlab # [    8.321513] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.863vm-test-run-nixbot-gitlab> gitlab # [    8.347506] systemd[1]: Finished Update is Completed.864vm-test-run-nixbot-gitlab> gitlab # [    8.350833] (udev-worker)[916]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.865vm-test-run-nixbot-gitlab> gitlab # [    8.354822] (udev-worker)[916]: Network interface NamePolicy= disabled on kernel command line.866vm-test-run-nixbot-gitlab> gitlab # [    8.362966] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.867vm-test-run-nixbot-gitlab> gitlab # [    8.390621] systemd[1]: Condition check resulted in Virtio network device being skipped.868vm-test-run-nixbot-gitlab> gitlab # [    8.392252] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore869vm-test-run-nixbot-gitlab> gitlab # [    8.395016] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.870vm-test-run-nixbot-gitlab> gitlab # [    8.397146] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67871vm-test-run-nixbot-gitlab> gitlab # [    8.400308] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore872vm-test-run-nixbot-gitlab> gitlab # [    8.402203] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki873vm-test-run-nixbot-gitlab> gitlab # [    8.403676] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki874vm-test-run-nixbot-gitlab> gitlab # [    8.433367] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.875vm-test-run-nixbot-gitlab> gitlab # [    8.434585] systemd[1]: Finished Create SUID/SGID Wrappers.876vm-test-run-nixbot-gitlab> gitlab # [    8.435480] systemd[1]: suid-sgid-wrappers.service: Consumed 211ms CPU time over 303ms wall clock time, 6.9M memory peak, 2.2M read from disk.877vm-test-run-nixbot-gitlab> gitlab # [    8.444797] mousedev: PS/2 mouse device common for all mice878vm-test-run-nixbot-gitlab> gitlab # [    8.658421] systemd[1]: Finished register-nix-paths.service.879vm-test-run-nixbot-gitlab> gitlab # [    8.659692] systemd[1]: Reached target System Initialization.880vm-test-run-nixbot-gitlab> gitlab # [    8.661076] systemd[1]: Started Discard unused filesystem blocks once a week.881vm-test-run-nixbot-gitlab> gitlab # [    8.662209] systemd[1]: Started logrotate.timer.882vm-test-run-nixbot-gitlab> gitlab # [    8.664219] systemd[1]: Started Daily Cleanup of Temporary Directories.883vm-test-run-nixbot-gitlab> gitlab # [    8.665262] systemd[1]: Reached target Timer Units.884vm-test-run-nixbot-gitlab> gitlab # [    8.666004] systemd[1]: Listening on D-Bus System Message Bus Socket.885vm-test-run-nixbot-gitlab> gitlab # [    8.666938] systemd[1]: Listening on Nix Daemon Socket.886vm-test-run-nixbot-gitlab> gitlab # [    8.680249] systemd[1]: Starting nixbot CI service socket...887vm-test-run-nixbot-gitlab> gitlab # [    8.681257] systemd[1]: Listening on Hostname Service Socket.888vm-test-run-nixbot-gitlab> gitlab # [    8.682119] systemd[1]: Listening on nixbot CI service socket.889vm-test-run-nixbot-gitlab> gitlab # [    8.682964] systemd[1]: Reached target Socket Units.890vm-test-run-nixbot-gitlab> gitlab # [    8.685205] systemd[1]: Starting D-Bus System Message Bus...891vm-test-run-nixbot-gitlab> gitlab # [    8.719597] dbus-broker-launch[1057]: Looking up NSS user entry for 'systemd-timesync'...892vm-test-run-nixbot-gitlab> gitlab # [    8.722924] dbus-broker-launch[1057]: NSS returned no entry for 'systemd-timesync'893vm-test-run-nixbot-gitlab> gitlab # [    8.724257] dbus-broker-launch[1057]: Invalid user-name in /nix/store/ifm9k575wb3j66gvjx1xkdmmknzqr8v8-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"894vm-test-run-nixbot-gitlab> gitlab # [    8.726848] systemd[1]: Started D-Bus System Message Bus.895vm-test-run-nixbot-gitlab> gitlab # [    8.740106] dbus-broker-launch[1057]: Ready896vm-test-run-nixbot-gitlab> gitlab # [    8.741284] systemd[1]: Reached target Basic System.897vm-test-run-nixbot-gitlab> gitlab # [    8.743618] systemd[1]: Started backdoor.service.898vm-test-run-nixbot-gitlab> gitlab # [    8.745691] systemd[1]: Starting gitlab-config.service...899vm-test-run-nixbot-gitlab> gitlab # [    8.751132] systemd[1]: Starting Import lastlog data into lastlog2 database...900vm-test-run-nixbot-gitlab> gitlab # [    8.763794] systemd[1]: Starting Logrotate configuration check...901vm-test-run-nixbot-gitlab> gitlab # [    8.768804] systemd[1]: Starting Name Service Cache Daemon (nsncd)...902vm-test-run-nixbot-gitlab> gitlab # [    8.772191] systemd[1]: Starting Post-Boot Actions...903vm-test-run-nixbot-gitlab> gitlab # [    8.783792] systemd[1]: Started Reset console on configuration changes.904vm-test-run-nixbot-gitlab> gitlab # [    8.785345] systemd[1]: Starting resolvconf update...905vm-test-run-nixbot-gitlab> gitlab # connecting to host...906vm-test-run-nixbot-gitlab> gitlab # [    8.790309] logrotate[1062]: warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.907vm-test-run-nixbot-gitlab> gitlab # [    8.794913] logrotate[1062]: reading config file /nix/store/hk6ay5j24mh4h8mpr5rxlxxxp8v61fnj-logrotate.conf908vm-test-run-nixbot-gitlab> gitlab # [    8.796502] logrotate[1062]: note: 'monthly' overrides previously specified 'weekly'909vm-test-run-nixbot-gitlab> gitlab # [    8.800156] logrotate[1062]: Reading state from file: /var/lib/logrotate.status910vm-test-run-nixbot-gitlab> gitlab # [    8.801559] logrotate[1062]: state file /var/lib/logrotate.status does not exist911vm-test-run-nixbot-gitlab> gitlab # [    8.805083] logrotate[1062]: Allocating hash table for state file, size 64 entries912vm-test-run-nixbot-gitlab> gitlab # [    8.806221] logrotate[1062]: Handling 4 logs913vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n'914vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell915vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.28 seconds)916vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.28 seconds)917vm-test-run-nixbot-gitlab> gitlab # [    8.806815] logrotate[1062]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed918vm-test-run-nixbot-gitlab> gitlab # [    8.812376] logrotate[1062]: considering log /var/log/btmp919vm-test-run-nixbot-gitlab> gitlab # [    8.813083] logrotate[1062]: Creating new state920vm-test-run-nixbot-gitlab> gitlab # [    8.813764] logrotate[1062]:   Now: 2026-07-04 10:49921vm-test-run-nixbot-gitlab> gitlab # [    8.814420] logrotate[1062]:   Last rotated at 2026-07-04 10:00922vm-test-run-nixbot-gitlab> gitlab # [    8.815192] logrotate[1062]:   log does not need rotating (log has already been rotated)923vm-test-run-nixbot-gitlab> gitlab # [    8.816928] logrotate[1062]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed924vm-test-run-nixbot-gitlab> gitlab # [    8.818915] logrotate[1062]: considering log /var/log/wtmp925vm-test-run-nixbot-gitlab> gitlab # [    8.819601] logrotate[1062]: Creating new state926vm-test-run-nixbot-gitlab> gitlab # [    8.820275] logrotate[1062]:   Now: 2026-07-04 10:49927vm-test-run-nixbot-gitlab> gitlab # [    8.820898] logrotate[1062]:   Last rotated at 2026-07-04 10:00928vm-test-run-nixbot-gitlab> gitlab # [    8.821630] logrotate[1062]:   log does not need rotating (log has already been rotated)929vm-test-run-nixbot-gitlab> gitlab # [    8.822601] logrotate[1062]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed930vm-test-run-nixbot-gitlab> gitlab # [    8.825112] logrotate[1062]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1062)931vm-test-run-nixbot-gitlab> gitlab # [    8.826255] logrotate[1062]: considering log /var/gitlab/state/log/*.log932vm-test-run-nixbot-gitlab> gitlab # [    8.827182] logrotate[1062]:   log /var/gitlab/state/log/*.log does not exist -- skipping933vm-test-run-nixbot-gitlab> gitlab # [    8.830190] logrotate[1062]: Creating new state934vm-test-run-nixbot-gitlab> gitlab # [    8.830866] logrotate[1062]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1062)935vm-test-run-nixbot-gitlab> gitlab # [    8.831984] logrotate[1062]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed936vm-test-run-nixbot-gitlab> gitlab # [    8.834122] logrotate[1062]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1062)937vm-test-run-nixbot-gitlab> gitlab # [    8.835324] logrotate[1062]: considering log /var/log/nginx/*.log938vm-test-run-nixbot-gitlab> gitlab # [    8.836916] logrotate[1062]:   log /var/log/nginx/*.log does not exist -- skipping939vm-test-run-nixbot-gitlab> gitlab # [    8.837950] logrotate[1062]: Creating new state940vm-test-run-nixbot-gitlab> gitlab # [    8.838586] logrotate[1062]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1062)941vm-test-run-nixbot-gitlab> gitlab # [    8.840227] nsncd[1063]: Jul 04 10:49:55.648 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"942vm-test-run-nixbot-gitlab> gitlab # [    8.842456] systemd[1]: Finished Logrotate configuration check.943vm-test-run-nixbot-gitlab> gitlab # [    8.844296] systemd[1]: Finished Post-Boot Actions.944vm-test-run-nixbot-gitlab> gitlab # [    8.845201] systemd[1]: Finished Import lastlog data into lastlog2 database.945vm-test-run-nixbot-gitlab> gitlab # [    8.846307] systemd[1]: Started Name Service Cache Daemon (nsncd).946vm-test-run-nixbot-gitlab> gitlab # [    8.847434] systemd[1]: Reached target Host and Network Name Lookups.947vm-test-run-nixbot-gitlab> gitlab # [    8.848541] systemd[1]: Reached target User and Group Name Lookups.948vm-test-run-nixbot-gitlab> gitlab # [    8.849419] systemd[1]: Starting User Login Management...949vm-test-run-nixbot-gitlab> gitlab # [    8.906029] systemd[1]: Stopped target Host and Network Name Lookups.950vm-test-run-nixbot-gitlab> gitlab # [    8.907613] systemd[1]: Stopping Host and Network Name Lookups...951vm-test-run-nixbot-gitlab> gitlab # [    8.908614] systemd[1]: Stopped target User and Group Name Lookups.952vm-test-run-nixbot-gitlab> gitlab # [    8.909596] systemd[1]: Stopping User and Group Name Lookups...953vm-test-run-nixbot-gitlab> gitlab # [    8.911039] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...954vm-test-run-nixbot-gitlab> gitlab # [    8.914608] systemd[1]: nscd.service: Deactivated successfully.955vm-test-run-nixbot-gitlab> gitlab # [    8.916381] systemd[1]: Stopped Name Service Cache Daemon (nsncd).956vm-test-run-nixbot-gitlab> gitlab # [    8.917292] systemd[1]: nscd.service: Consumed 22ms CPU time over 143ms wall clock time, 8.3M memory peak, 5.9M read from disk.957vm-test-run-nixbot-gitlab> gitlab # [    8.919193] systemd-logind[1101]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)958vm-test-run-nixbot-gitlab> gitlab # [    8.922794] systemd-logind[1101]: Watching system buttons on /dev/input/event0 (gpio-keys)959vm-test-run-nixbot-gitlab> gitlab # [    8.923978] systemd-logind[1101]: New seat seat0.960vm-test-run-nixbot-gitlab> gitlab # [    8.924772] systemd[1]: Started User Login Management.961vm-test-run-nixbot-gitlab> gitlab # [    8.931415] systemd[1]: Starting linger-users.service...962vm-test-run-nixbot-gitlab> gitlab # [    8.932367] systemd[1]: Starting Name Service Cache Daemon (nsncd)...963vm-test-run-nixbot-gitlab> gitlab # [    8.954625] systemd[1]: Finished resolvconf update.964vm-test-run-nixbot-gitlab> gitlab # [    8.955823] systemd[1]: Reached target Preparation for Network.965vm-test-run-nixbot-gitlab> gitlab # [    8.960428] systemd[1]: Starting DHCP Client...966vm-test-run-nixbot-gitlab> gitlab # [    8.963519] systemd[1]: Starting Address configuration of eth1...967vm-test-run-nixbot-gitlab> gitlab # [    8.966765] nsncd[1152]: Jul 04 10:49:55.795 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"968vm-test-run-nixbot-gitlab> gitlab # [    8.972873] systemd[1]: Starting Extra networking commands....969vm-test-run-nixbot-gitlab> gitlab # [    8.973745] systemd[1]: Started Name Service Cache Daemon (nsncd).970vm-test-run-nixbot-gitlab> gitlab # [    8.974752] systemd[1]: linger-users.service: Deactivated successfully.971vm-test-run-nixbot-gitlab> gitlab # [    8.975976] systemd[1]: Finished linger-users.service.972vm-test-run-nixbot-gitlab> gitlab # [    8.979137] systemd[1]: Reached target Host and Network Name Lookups.973vm-test-run-nixbot-gitlab> gitlab # [    8.980689] systemd[1]: Reached target User and Group Name Lookups.974vm-test-run-nixbot-gitlab> gitlab # [    9.013937] network-addresses-eth1-start[1169]: adding address 192.168.1.1/24... done975vm-test-run-nixbot-gitlab> gitlab # [    9.018736] network-addresses-eth1-start[1169]: adding address 2001:db8:1::1/64... done976vm-test-run-nixbot-gitlab> gitlab # [    9.053631] systemd[1]: Finished Address configuration of eth1.977vm-test-run-nixbot-gitlab> gitlab # [    9.100912] systemd[1]: Finished Extra networking commands..978vm-test-run-nixbot-gitlab> gitlab # [    9.116379] systemd[1]: Reached target Network.979vm-test-run-nixbot-gitlab> gitlab # [    9.159564] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:08.0/virtio7/input/input3980vm-test-run-nixbot-gitlab> gitlab # [    9.186501] systemd[1]: Starting gitlab-workhorse.service...981vm-test-run-nixbot-gitlab> gitlab # [    9.190593] dhcpcd[1207]: dhcpcd-10.3.2 starting982vm-test-run-nixbot-gitlab> gitlab # [    9.194752] dhcpcd[1236]: dev: loaded udev983vm-test-run-nixbot-gitlab> gitlab # [    9.230473] systemd[1]: Starting Nginx Web Server...984vm-test-run-nixbot-gitlab> gitlab # [    9.247653] systemd[1]: Starting PostgreSQL Server...985vm-test-run-nixbot-gitlab> gitlab # [    9.254349] 8021q: 802.1Q VLAN Support v1.8986vm-test-run-nixbot-gitlab> gitlab # [    9.254731] 8021q: adding VLAN 0 to HW filter on device eth1987vm-test-run-nixbot-gitlab> gitlab # [    9.254182] systemd[1]: Starting Redis Server - redis-gitlab...988vm-test-run-nixbot-gitlab> gitlab # [    9.302506] systemd[1]: Starting Permit User Sessions...989vm-test-run-nixbot-gitlab> gitlab # [    9.316289] cfg80211: Loading compiled-in X.509 certificates for regulatory database990vm-test-run-nixbot-gitlab> gitlab # [    9.325963] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'991vm-test-run-nixbot-gitlab> gitlab # [    9.326463] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'992vm-test-run-nixbot-gitlab> gitlab # [    9.328031] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2993vm-test-run-nixbot-gitlab> gitlab # [    9.328347] cfg80211: failed to load regulatory.db994vm-test-run-nixbot-gitlab> gitlab # [    9.335153] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.995vm-test-run-nixbot-gitlab> gitlab # [    9.339830] systemd[1]: Finished Permit User Sessions.996vm-test-run-nixbot-gitlab> gitlab # [    9.350382] systemd-logind[1101]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)997vm-test-run-nixbot-gitlab> gitlab # [    9.351805] systemd[1]: Started Getty on tty1.998vm-test-run-nixbot-gitlab> gitlab # [    9.356352] systemd[1]: Reached target Login Prompts.999vm-test-run-nixbot-gitlab> gitlab # [    9.357298] systemd[1]: Starting Virtual Console Setup...1000vm-test-run-nixbot-gitlab> gitlab # [    9.401238] 8021q: adding VLAN 0 to HW filter on device eth01001vm-test-run-nixbot-gitlab> gitlab # [    9.399189] dhcpcd[1236]: eth0: waiting for carrier1002vm-test-run-nixbot-gitlab> gitlab # [    9.400790] dhcpcd[1236]: eth0: carrier acquired1003vm-test-run-nixbot-gitlab> gitlab # [    9.404558] dhcpcd[1236]: DUID 00:01:00:01:31:db:a1:d4:52:54:00:12:34:561004vm-test-run-nixbot-gitlab> gitlab # [    9.405973] dhcpcd[1236]: eth0: IAID 00:12:34:561005vm-test-run-nixbot-gitlab> gitlab # [    9.407286] dhcpcd[1236]: eth0: adding address fe80::5054:ff:fe12:34561006vm-test-run-nixbot-gitlab> gitlab # [    9.436652] systemd-vconsole-setup[1262]: Configuration of first virtual console was skipped, ignoring remaining ones.1007vm-test-run-nixbot-gitlab> gitlab # [    9.440271] systemd[1]: Finished Virtual Console Setup.1008vm-test-run-nixbot-gitlab> gitlab # [    9.497141] nginx-pre-start[1276]: nginx: the configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf syntax is ok1009vm-test-run-nixbot-gitlab> gitlab # [    9.500810] nginx-pre-start[1276]: nginx: configuration file /nix/store/2526shpjv0bzachwf4qi968gxiiyv4kq-nginx.conf test is successful1010vm-test-run-nixbot-gitlab> gitlab # [    9.503924] systemd[1]: Started Nginx Web Server.1011vm-test-run-nixbot-gitlab> gitlab # [    9.505215] redis-server[1278]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.1012vm-test-run-nixbot-gitlab> gitlab # [    9.507304] redis-server[1278]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1013vm-test-run-nixbot-gitlab> gitlab # [    9.512502] redis-server[1278]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1278, just started1014vm-test-run-nixbot-gitlab> gitlab # [    9.514500] redis-server[1278]: Configuration loaded1015vm-test-run-nixbot-gitlab> gitlab # [    9.516306] postgresql-pre-start[1275]: The files belonging to this database system will be owned by user "postgres".1016vm-test-run-nixbot-gitlab> gitlab # [    9.518030] postgresql-pre-start[1275]: This user must also own the server process.1017vm-test-run-nixbot-gitlab> gitlab # [    9.519196] systemd[1]: Started Redis Server - redis-gitlab.1018vm-test-run-nixbot-gitlab> gitlab # [    9.522290] redis[1278]: monotonic clock: ARM CNTVCT @ 1000 ticks/us1019vm-test-run-nixbot-gitlab> gitlab # [    9.523542] postgresql-pre-start[1275]: The database cluster will be initialized with locale "en_US.UTF-8".1020vm-test-run-nixbot-gitlab> gitlab # [    9.525533] postgresql-pre-start[1275]: The default database encoding has accordingly been set to "UTF8".1021vm-test-run-nixbot-gitlab> gitlab # [    9.527125] postgresql-pre-start[1275]: The default text search configuration will be set to "english".1022vm-test-run-nixbot-gitlab> gitlab # [    9.528780] postgresql-pre-start[1275]: Data page checksums are disabled.1023vm-test-run-nixbot-gitlab> gitlab # [    9.529927] postgresql-pre-start[1275]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok1024vm-test-run-nixbot-gitlab> gitlab # [    9.531548] postgresql-pre-start[1275]: creating subdirectories ... ok1025vm-test-run-nixbot-gitlab> gitlab # [    9.532687] postgresql-pre-start[1275]: selecting dynamic shared memory implementation ... posix1026vm-test-run-nixbot-gitlab> gitlab # [    9.533962] redis[1278]: Running mode=standalone, port=0.1027vm-test-run-nixbot-gitlab> gitlab # [    9.535150] redis[1278]: Server initialized1028vm-test-run-nixbot-gitlab> gitlab # [    9.536372] redis[1278]: Ready to accept connections unix1029vm-test-run-nixbot-gitlab> gitlab # [    9.537621] redis[1278]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1030vm-test-run-nixbot-gitlab> gitlab # [    9.556225] postgresql-pre-start[1275]: selecting default "max_connections" ... 1001031vm-test-run-nixbot-gitlab> gitlab # [    9.605585] postgresql-pre-start[1275]: selecting default "shared_buffers" ... 128MB1032vm-test-run-nixbot-gitlab> gitlab # [    9.834156] systemd[1]: Started gitlab-workhorse.service.1033vm-test-run-nixbot-gitlab> gitlab # [   10.040087] postgresql-pre-start[1275]: selecting default time zone ... UTC1034vm-test-run-nixbot-gitlab> gitlab # [   10.043935] postgresql-pre-start[1275]: creating configuration files ... ok1035vm-test-run-nixbot-gitlab> gitlab # [   10.108927] gitlab-workhorse[1302]: time="2026-07-04T10:49:56Z" level=info msg=Starting build_time=19700101.000000 version=18.11.41036vm-test-run-nixbot-gitlab> gitlab # [   10.111553] gitlab-workhorse[1302]: time="2026-07-04T10:49:56Z" level=info msg="Using redis/go-redis"1037vm-test-run-nixbot-gitlab> gitlab # [   10.113756] gitlab-workhorse[1302]: time="2026-07-04T10:49:56Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1038vm-test-run-nixbot-gitlab> gitlab # [   10.261939] postgresql-pre-start[1275]: running bootstrap script ... ok1039vm-test-run-nixbot-gitlab> gitlab # [   10.728138] dhcpcd[1236]: eth0: soliciting a DHCP lease1040vm-test-run-nixbot-gitlab> gitlab # [   10.736886] dhcpcd[1236]: eth0: offered 10.0.2.15 from 10.0.2.21041vm-test-run-nixbot-gitlab> gitlab # [   10.760566] dhcpcd[1236]: eth0: probing address 10.0.2.15/241042vm-test-run-nixbot-gitlab> gitlab # [   10.980607] postgresql-pre-start[1275]: performing post-bootstrap initialization ... ok1043vm-test-run-nixbot-gitlab> gitlab # [   11.219613] postgresql-pre-start[1275]: syncing data to disk ... ok1044vm-test-run-nixbot-gitlab> gitlab # [   11.221287] postgresql-pre-start[1275]: initdb: warning: enabling "trust" authentication for local connections1045vm-test-run-nixbot-gitlab> gitlab # [   11.223541] postgresql-pre-start[1275]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1046vm-test-run-nixbot-gitlab> gitlab # [   11.227184] postgresql-pre-start[1275]: Success. You can now start the database server using:1047vm-test-run-nixbot-gitlab> gitlab # [   11.229186] postgresql-pre-start[1275]:     pg_ctl -D /var/lib/postgresql/17 -l logfile start1048vm-test-run-nixbot-gitlab> gitlab # [   11.411215] postgres[1336]: [1336] LOG:  starting PostgreSQL 17.10 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit1049vm-test-run-nixbot-gitlab> gitlab # [   11.417228] postgres[1336]: [1336] LOG:  listening on IPv6 address "::1", port 54321050vm-test-run-nixbot-gitlab> gitlab # [   11.420269] postgres[1336]: [1336] LOG:  listening on IPv4 address "127.0.0.1", port 54321051vm-test-run-nixbot-gitlab> gitlab # [   11.423679] postgres[1336]: [1336] LOG:  listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1052vm-test-run-nixbot-gitlab> gitlab # [   11.427775] postgres[1347]: [1347] LOG:  database system was shut down at 2026-07-04 10:49:57 GMT1053vm-test-run-nixbot-gitlab> gitlab # [   11.431406] postgres[1336]: [1336] LOG:  database system is ready to accept connections1054vm-test-run-nixbot-gitlab> gitlab # [   11.435830] systemd[1]: Started PostgreSQL Server.1055vm-test-run-nixbot-gitlab> gitlab # [   11.439747] systemd[1]: Starting PostgreSQL Setup Scripts...1056vm-test-run-nixbot-gitlab> gitlab # [   11.684666] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1344]: mkdir -p /var/gitlab/state/home/.ssh: OK1057vm-test-run-nixbot-gitlab> gitlab # [   11.688644] c0sj6d459nhf3wg9pm6ziy71l4581qq8-gitlab-config[1344]: chmod 700 /var/gitlab/state/home/.ssh: OK1058vm-test-run-nixbot-gitlab> gitlab # [   11.692755] postgresql-setup-start[1359]: CREATE DATABASE1059vm-test-run-nixbot-gitlab> gitlab # [   11.739824] postgresql-setup-start[1370]: CREATE ROLE1060vm-test-run-nixbot-gitlab> gitlab # [   11.763311] postgresql-setup-start[1375]: ALTER DATABASE1061vm-test-run-nixbot-gitlab> gitlab # [   11.780495] systemd[1]: Finished gitlab-config.service.1062vm-test-run-nixbot-gitlab> gitlab # [   11.785494] systemd[1]: Started gitaly.service.1063vm-test-run-nixbot-gitlab> gitlab # [   11.793988] dhcpcd[1236]: eth0: soliciting an IPv6 router1064vm-test-run-nixbot-gitlab> gitlab # [   11.796756] dhcpcd[1236]: eth0: Router Advertisement from fe80::21065vm-test-run-nixbot-gitlab> gitlab # [   11.799426] dhcpcd[1236]: eth0: adding address fec0::5054:ff:fe12:3456/641066vm-test-run-nixbot-gitlab> gitlab # [   11.802412] dhcpcd[1236]: eth0: adding route to fec0::/641067vm-test-run-nixbot-gitlab> gitlab # [   11.804872] dhcpcd[1236]: eth0: adding default route via fe80::21068vm-test-run-nixbot-gitlab> gitlab # [   11.808427] postgresql-setup-start[1384]: CREATE ROLE1069vm-test-run-nixbot-gitlab> gitlab # [   11.812933] systemd[1]: Finished PostgreSQL Setup Scripts.1070vm-test-run-nixbot-gitlab> gitlab # [   11.815668] systemd[1]: Reached target PostgreSQL.1071vm-test-run-nixbot-gitlab> gitlab # [   11.821142] systemd[1]: Starting gitlab-db-config.service...1072vm-test-run-nixbot-gitlab> gitlab # [   11.828134] systemd[1]: Starting gitlab-postgresql.service...1073vm-test-run-nixbot-gitlab> gitlab # [   12.007812] gitlab-postgresql-start[1395]: CREATE DATABASE1074vm-test-run-nixbot-gitlab> gitlab # [   12.085688] gitlab-postgresql-start[1405]: CREATE EXTENSION1075vm-test-run-nixbot-gitlab> gitlab # [   12.139116] gitlab-postgresql-start[1412]: CREATE EXTENSION1076vm-test-run-nixbot-gitlab> gitlab # [   12.142782] systemd[1]: Finished gitlab-postgresql.service.1077vm-test-run-nixbot-gitlab> gitlab # [   12.556625] gitaly[1383]: time="2026-07-04T10:49:59.385Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/krygc3bmd623znp2pddh3fv21q5f3nz2-gitaly.toml' and fix the errors that are printed." pid=13831078vm-test-run-nixbot-gitlab> gitlab # [   12.568735] gitaly[1383]: time="2026-07-04T10:49:59.386Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=13831079vm-test-run-nixbot-gitlab> gitlab # [   12.570412] gitaly[1383]: time="2026-07-04T10:49:59.386Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=13831080vm-test-run-nixbot-gitlab> gitlab # [   12.572815] gitaly[1383]: time="2026-07-04T10:49:59.386Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=13831081vm-test-run-nixbot-gitlab> gitlab # [   12.575077] gitaly[1383]: time="2026-07-04T10:49:59.386Z" level=info msg="Starting Gitaly" pid=1383 version=18.11.41082vm-test-run-nixbot-gitlab> gitlab # [   12.776210] gitaly[1383]: time="2026-07-04T10:49:59.607Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=219 pid=13831083vm-test-run-nixbot-gitlab> gitlab # [   12.778636] gitaly[1383]: time="2026-07-04T10:49:59.607Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=13831084vm-test-run-nixbot-gitlab> gitlab # [   12.780968] gitaly[1383]: time="2026-07-04T10:49:59.608Z" level=info msg="finished initializing command factory" duration_ms=0 pid=13831085vm-test-run-nixbot-gitlab> gitlab # [   12.783176] gitaly[1383]: time="2026-07-04T10:49:59.608Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1383/git-exec-2802010508.d/git pid=13831086vm-test-run-nixbot-gitlab> gitlab # [   12.785997] gitaly[1383]: time="2026-07-04T10:49:59.612Z" level=info msg="finished detecting git version" duration_ms=4 pid=13831087vm-test-run-nixbot-gitlab> gitlab # [   12.788113] gitaly[1383]: time="2026-07-04T10:49:59.612Z" level=info msg="using Git version" pid=1383 version=2.53.g1088vm-test-run-nixbot-gitlab> gitlab # [   12.790263] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="clearing disk cache object folder" pid=1383 storage=default1089vm-test-run-nixbot-gitlab> gitlab # [   12.792943] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3770393269 pid=1383 storage=default1090vm-test-run-nixbot-gitlab> gitlab # [   12.798059] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3770393269 pid=1383 storage=default1091vm-test-run-nixbot-gitlab> gitlab # [   12.802377] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=13831092vm-test-run-nixbot-gitlab> gitlab # [   12.806204] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=13831093vm-test-run-nixbot-gitlab> gitlab # [   12.810250] gitaly[1383]: time="2026-07-04T10:49:59.613Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3770393269 clear_duration_ms=0 pid=1383 storage=default1094vm-test-run-nixbot-gitlab> gitlab # [   12.814385] gitaly[1383]: time="2026-07-04T10:49:59.627Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=13831095vm-test-run-nixbot-gitlab> gitlab # [   12.817598] gitaly[1383]: time="2026-07-04T10:49:59.627Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1383 schema=unix1096vm-test-run-nixbot-gitlab> gitlab # [   12.820652] gitaly[1383]: time="2026-07-04T10:49:59.627Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1383/sock.d/intern pid=1383 schema=unix1097vm-test-run-nixbot-gitlab> gitlab # [   12.823621] gitaly[1383]: time="2026-07-04T10:49:59.628Z" level=info msg="finished tempdir cleaner walk" pid=1383 storage=default time_ms=01098vm-test-run-nixbot-gitlab> gitlab # [   12.826402] gitaly[1383]: time="2026-07-04T10:49:59.628Z" level=info msg="counting repositories" pid=13831099vm-test-run-nixbot-gitlab> gitlab # [   12.828421] gitaly[1383]: time="2026-07-04T10:49:59.628Z" level=info msg="starting to count repositories" pid=1383 storage_path=/var/gitlab/state/repositories1100vm-test-run-nixbot-gitlab> gitlab # [   12.831288] gitaly[1383]: time="2026-07-04T10:49:59.628Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1383 storage_path=/var/gitlab/state/repositories1101vm-test-run-nixbot-gitlab> gitlab # [   12.834849] gitaly[1383]: time="2026-07-04T10:49:59.628Z" level=info msg="completed counting all repositories" counting_duration=0 pid=13831102vm-test-run-nixbot-gitlab> gitlab # [   12.837673] gitaly[1383]: time="2026-07-04T10:49:59.630Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=13831103vm-test-run-nixbot-gitlab> gitlab # [   12.840384] gitaly[1383]: time="2026-07-04T10:49:59.630Z" level=info msg="Started Gitaly" duration_ms=243 pid=13831104vm-test-run-nixbot-gitlab> gitlab # [   12.842603] gitaly[1383]: time="2026-07-04T10:49:59.630Z" level=info msg="maintenance: daily scheduled" pid=1383 scheduled="2026-07-04 12:00:00 +0000 UTC"1105vm-test-run-nixbot-gitlab> gitlab # [   13.060409] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1106vm-test-run-nixbot-gitlab> gitlab # [   13.070794] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1107vm-test-run-nixbot-gitlab> gitlab # [   13.079974] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1390]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1108vm-test-run-nixbot-gitlab> gitlab # [   16.143362] gitaly[1383]: time="2026-07-04T10:50:02.973Z" level=info msg="License database preloaded" duration_ms=3587 pid=13831109vm-test-run-nixbot-gitlab> gitlab # [   16.321639] dhcpcd[1236]: eth0: leased 10.0.2.15 for 86400 seconds1110vm-test-run-nixbot-gitlab> gitlab # [   16.324538] dhcpcd[1236]: eth0: adding route to 10.0.2.0/241111vm-test-run-nixbot-gitlab> gitlab # [   16.326906] dhcpcd[1236]: eth0: adding default route via 10.0.2.21112vm-test-run-nixbot-gitlab> gitlab # [   16.488544] systemd[1]: Started DHCP Client.1113vm-test-run-nixbot-gitlab> gitlab # [   16.490597] systemd[1]: Reached target Network is Online.1114vm-test-run-nixbot-gitlab> gitlab # [   54.319851] postgres[1841]: [1841] ERROR:  relation "feature_gates" does not exist at character 5231115vm-test-run-nixbot-gitlab> gitlab # [   54.321462] postgres[1841]: [1841] STATEMENT:  SELECT a.attname, format_type(a.atttypid, a.atttypmod),1116vm-test-run-nixbot-gitlab> gitlab # [   54.322719] postgres[1841]: 	       pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1117vm-test-run-nixbot-gitlab> gitlab # [   54.324007] postgres[1841]: 	       c.collname, col_description(a.attrelid, a.attnum) AS comment,1118vm-test-run-nixbot-gitlab> gitlab # [   54.325222] postgres[1841]: 	       attidentity AS identity,1119vm-test-run-nixbot-gitlab> gitlab # [   54.326073] postgres[1841]: 	       attgenerated as attgenerated1120vm-test-run-nixbot-gitlab> gitlab # [   54.326911] postgres[1841]: 	  FROM pg_attribute a1121vm-test-run-nixbot-gitlab> gitlab # [   54.327658] postgres[1841]: 	  LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1122vm-test-run-nixbot-gitlab> gitlab # [   54.328920] postgres[1841]: 	  LEFT JOIN pg_type t ON a.atttypid = t.oid1123vm-test-run-nixbot-gitlab> gitlab # [   54.329831] postgres[1841]: 	  LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1124vm-test-run-nixbot-gitlab> gitlab # [   54.331305] postgres[1841]: 	 WHERE a.attrelid = '"feature_gates"'::regclass1125vm-test-run-nixbot-gitlab> gitlab # [   54.332370] postgres[1841]: 	   AND a.attnum > 0 AND NOT a.attisdropped1126vm-test-run-nixbot-gitlab> gitlab # [   54.333278] postgres[1841]: 	 ORDER BY a.attnum1127vm-test-run-nixbot-gitlab> gitlab # [   54.333930] postgres[1841]:1128vm-test-run-nixbot-gitlab> gitlab # [   54.653989] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1390]: W, [2026-07-04T10:50:41.484512 #1390]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1129vm-test-run-nixbot-gitlab> gitlab # [   60.190752] gitlab-workhorse[1302]: time="2026-07-04T10:50:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1130vm-test-run-nixbot-gitlab> gitlab # [   62.519633] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1923]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE:  extension "btree_gist" already exists, skipping1131vm-test-run-nixbot-gitlab> gitlab # [   62.520163] 9831d0wkmwrjcjk9y25pj7qfsr5zj2vi-gitlab-db-config[1923]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE:  extension "pg_trgm" already exists, skipping1132vm-test-run-nixbot-gitlab> gitlab # [   70.201093] gitlab-workhorse[1302]: time="2026-07-04T10:50:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1133vm-test-run-nixbot-gitlab> gitlab # [   80.210427] gitlab-workhorse[1302]: time="2026-07-04T10:51:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1134vm-test-run-nixbot-gitlab> gitlab # [   86.271936] postgres[2017]: [2017] ERROR:  relation "table_without_models" does not exist1135vm-test-run-nixbot-gitlab> gitlab # [   86.272511] postgres[2017]: [2017] STATEMENT:  SELECT pg_get_serial_sequence('table_without_models', 'id')1136vm-test-run-nixbot-gitlab> gitlab # [   90.218031] gitlab-workhorse[1302]: time="2026-07-04T10:51:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1137vm-test-run-nixbot-gitlab> gitlab # [  100.228274] gitlab-workhorse[1302]: time="2026-07-04T10:51:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1138vm-test-run-nixbot-gitlab> gitlab # [  110.229413] gitlab-workhorse[1302]: time="2026-07-04T10:51:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1139vm-test-run-nixbot-gitlab> gitlab # [  116.645106] systemd[1]: Finished gitlab-db-config.service.1140vm-test-run-nixbot-gitlab> gitlab # [  116.652095] systemd[1]: Started gitlab-sidekiq.service.1141vm-test-run-nixbot-gitlab> gitlab # [  116.656574] systemd[1]: Starting gitlab.service...1142vm-test-run-nixbot-gitlab> gitlab # [  117.180562] bundle[2383]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1143vm-test-run-nixbot-gitlab> gitlab # [  117.187067] bundle[2383]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1144vm-test-run-nixbot-gitlab> gitlab # [  117.193229] bundle[2383]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1145vm-test-run-nixbot-gitlab> gitlab # [  120.238251] gitlab-workhorse[1302]: time="2026-07-04T10:51:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146vm-test-run-nixbot-gitlab> gitlab # [  121.725414] sidekiq-cluster[2382]: {"severity":"INFO","time":"2026-07-04T10:51:48.526Z","message":"Starting cluster with 1 processes"}1147vm-test-run-nixbot-gitlab> gitlab # [  121.745718] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.543Z","pid":2383,"message":"* Enabling systemd notification integration"}1148vm-test-run-nixbot-gitlab> gitlab # [  121.747428] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"Puma starting in cluster mode..."}1149vm-test-run-nixbot-gitlab> gitlab # [  121.748959] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"* Puma version: 7.2.0 (\"On The Corner\")"}1150vm-test-run-nixbot-gitlab> gitlab # [  121.750520] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1151vm-test-run-nixbot-gitlab> gitlab # [  121.752482] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*  Min threads: 0"}1152vm-test-run-nixbot-gitlab> gitlab # [  121.753768] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*  Max threads: 4"}1153vm-test-run-nixbot-gitlab> gitlab # [  121.755056] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*  Environment: production"}1154vm-test-run-nixbot-gitlab> gitlab # [  121.756468] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*   Master PID: 2383"}1155vm-test-run-nixbot-gitlab> gitlab # [  121.757776] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*      Workers: 2"}1156vm-test-run-nixbot-gitlab> gitlab # [  121.759052] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"*     Restarts: (✔) hot (✖) phased (✖) refork"}1157vm-test-run-nixbot-gitlab> gitlab # [  121.760719] bundle[2383]: {"timestamp":"2026-07-04T10:51:48.577Z","pid":2383,"message":"* Preloading application"}1158vm-test-run-nixbot-gitlab> gitlab # [  130.240276] gitlab-workhorse[1302]: time="2026-07-04T10:51:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1159vm-test-run-nixbot-gitlab> gitlab # [  140.242552] gitlab-workhorse[1302]: time="2026-07-04T10:52:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1160vm-test-run-nixbot-gitlab> gitlab # [  150.243715] gitlab-workhorse[1302]: time="2026-07-04T10:52:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1161vm-test-run-nixbot-gitlab> gitlab # [  158.184956] bundle[2383]: W, [2026-07-04T10:52:25.016081 #2383]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1162vm-test-run-nixbot-gitlab> gitlab # [  160.244943] gitlab-workhorse[1302]: time="2026-07-04T10:52:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1163vm-test-run-nixbot-gitlab> gitlab # [  162.634342] sidekiq-cluster[2431]: 2026-07-04T10:52:29.464Z pid=2431 tid=ttj INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2431, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>450510848}1164vm-test-run-nixbot-gitlab> gitlab # [  162.924122] sidekiq-cluster[2431]: W, [2026-07-04T10:52:29.755477 #2431]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1165vm-test-run-nixbot-gitlab> gitlab # [  162.984480] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:29.816Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1166vm-test-run-nixbot-gitlab> gitlab # [  166.282305] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.112Z","message":"GitLab reliable fetch activated!"}1167vm-test-run-nixbot-gitlab> gitlab # [  166.282675] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.112Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1168vm-test-run-nixbot-gitlab> gitlab # [  166.316114] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.147Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1169vm-test-run-nixbot-gitlab> gitlab # [  166.323154] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.151Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1170vm-test-run-nixbot-gitlab> gitlab # [  166.330185] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.154Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1171vm-test-run-nixbot-gitlab> gitlab # [  166.340233] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.157Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1172vm-test-run-nixbot-gitlab> gitlab # [  166.342529] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.160Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1173vm-test-run-nixbot-gitlab> gitlab # [  166.345115] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.163Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1174vm-test-run-nixbot-gitlab> gitlab # [  166.348370] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.166Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1175vm-test-run-nixbot-gitlab> gitlab # [  166.354929] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.169Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1176vm-test-run-nixbot-gitlab> gitlab # [  166.361590] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.171Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"}1177vm-test-run-nixbot-gitlab> gitlab # [  166.368393] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.185Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1178vm-test-run-nixbot-gitlab> gitlab # [  166.376909] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.187Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1179vm-test-run-nixbot-gitlab> gitlab # [  166.383532] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.190Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1180vm-test-run-nixbot-gitlab> gitlab # [  166.389546] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.192Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1181vm-test-run-nixbot-gitlab> gitlab # [  166.395963] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.194Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1182vm-test-run-nixbot-gitlab> gitlab # [  166.402503] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.196Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1183vm-test-run-nixbot-gitlab> gitlab # [  166.408187] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.197Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"}1184vm-test-run-nixbot-gitlab> gitlab # [  166.413152] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.199Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1185vm-test-run-nixbot-gitlab> gitlab # [  166.417509] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.202Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1186vm-test-run-nixbot-gitlab> gitlab # [  166.421666] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.205Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1187vm-test-run-nixbot-gitlab> gitlab # [  166.427121] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.207Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1188vm-test-run-nixbot-gitlab> gitlab # [  166.431426] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.211Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1189vm-test-run-nixbot-gitlab> gitlab # [  166.436165] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.213Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1190vm-test-run-nixbot-gitlab> gitlab # [  166.440949] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.216Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1191vm-test-run-nixbot-gitlab> gitlab # [  166.445167] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.219Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1192vm-test-run-nixbot-gitlab> gitlab # [  166.449481] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.221Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1193vm-test-run-nixbot-gitlab> gitlab # [  166.453610] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.224Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1194vm-test-run-nixbot-gitlab> gitlab # [  166.457320] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.227Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1195vm-test-run-nixbot-gitlab> gitlab # [  166.460748] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.229Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1196vm-test-run-nixbot-gitlab> gitlab # [  166.464125] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.232Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1197vm-test-run-nixbot-gitlab> gitlab # [  166.467446] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.234Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1198vm-test-run-nixbot-gitlab> gitlab # [  166.470903] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.237Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1199vm-test-run-nixbot-gitlab> gitlab # [  166.473962] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.239Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1200vm-test-run-nixbot-gitlab> gitlab # [  166.477189] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.242Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1201vm-test-run-nixbot-gitlab> gitlab # [  166.479912] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.244Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1202vm-test-run-nixbot-gitlab> gitlab # [  166.482960] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.247Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1203vm-test-run-nixbot-gitlab> gitlab # [  166.485785] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.249Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1204vm-test-run-nixbot-gitlab> gitlab # [  166.488437] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.251Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1205vm-test-run-nixbot-gitlab> gitlab # [  166.490983] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.253Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1206vm-test-run-nixbot-gitlab> gitlab # [  166.493441] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.255Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1207vm-test-run-nixbot-gitlab> gitlab # [  166.495754] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.257Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1208vm-test-run-nixbot-gitlab> gitlab # [  166.498167] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.259Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1209vm-test-run-nixbot-gitlab> gitlab # [  166.500636] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.261Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1210vm-test-run-nixbot-gitlab> gitlab # [  166.502993] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.263Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1211vm-test-run-nixbot-gitlab> gitlab # [  166.505439] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.265Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1212vm-test-run-nixbot-gitlab> gitlab # [  166.507616] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.267Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1213vm-test-run-nixbot-gitlab> gitlab # [  166.510062] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.269Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1214vm-test-run-nixbot-gitlab> gitlab # [  166.512468] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.271Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1215vm-test-run-nixbot-gitlab> gitlab # [  166.514684] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.273Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1216vm-test-run-nixbot-gitlab> gitlab # [  166.516924] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.276Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1217vm-test-run-nixbot-gitlab> gitlab # [  166.519062] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.278Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1218vm-test-run-nixbot-gitlab> gitlab # [  166.521306] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.281Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1219vm-test-run-nixbot-gitlab> gitlab # [  166.523442] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.283Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1220vm-test-run-nixbot-gitlab> gitlab # [  166.525659] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.286Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1221vm-test-run-nixbot-gitlab> gitlab # [  166.527807] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.344Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1222vm-test-run-nixbot-gitlab> gitlab # [  166.530402] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.348Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1223vm-test-run-nixbot-gitlab> gitlab # [  166.532595] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.350Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1224vm-test-run-nixbot-gitlab> gitlab # [  166.534728] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.353Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1225vm-test-run-nixbot-gitlab> gitlab # [  166.536913] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.355Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1226vm-test-run-nixbot-gitlab> gitlab # [  166.539217] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.359Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1227vm-test-run-nixbot-gitlab> gitlab # [  166.541589] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.361Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1228vm-test-run-nixbot-gitlab> gitlab # [  166.543845] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.365Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1229vm-test-run-nixbot-gitlab> gitlab # [  166.546050] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.367Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1230vm-test-run-nixbot-gitlab> gitlab # [  166.548123] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.370Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1231vm-test-run-nixbot-gitlab> gitlab # [  166.550289] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.374Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1232vm-test-run-nixbot-gitlab> gitlab # [  166.552534] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.377Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1233vm-test-run-nixbot-gitlab> gitlab # [  166.554796] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.381Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1234vm-test-run-nixbot-gitlab> gitlab # [  166.557097] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.383Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1235vm-test-run-nixbot-gitlab> gitlab # [  166.559207] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.385Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1236vm-test-run-nixbot-gitlab> gitlab # [  166.561494] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.387Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1237vm-test-run-nixbot-gitlab> gitlab # [  166.563799] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.390Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1238vm-test-run-nixbot-gitlab> gitlab # [  166.566270] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.391Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1239vm-test-run-nixbot-gitlab> gitlab # [  166.568563] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.394Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1240vm-test-run-nixbot-gitlab> gitlab # [  166.570764] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.395Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1241vm-test-run-nixbot-gitlab> gitlab # [  166.573030] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.397Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1242vm-test-run-nixbot-gitlab> gitlab # [  166.575344] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.400Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1243vm-test-run-nixbot-gitlab> gitlab # [  166.577757] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.401Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1244vm-test-run-nixbot-gitlab> gitlab # [  166.580099] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.404Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1245vm-test-run-nixbot-gitlab> gitlab # [  166.582365] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.406Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1246vm-test-run-nixbot-gitlab> gitlab # [  166.584949] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.408Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1247vm-test-run-nixbot-gitlab> gitlab # [  166.587177] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.410Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1248vm-test-run-nixbot-gitlab> gitlab # [  166.589477] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.412Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1249vm-test-run-nixbot-gitlab> gitlab # [  166.591848] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.414Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1250vm-test-run-nixbot-gitlab> gitlab # [  166.594455] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.416Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1251vm-test-run-nixbot-gitlab> gitlab # [  166.596859] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.419Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1252vm-test-run-nixbot-gitlab> gitlab # [  166.599125] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.421Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1253vm-test-run-nixbot-gitlab> gitlab # [  166.601622] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.424Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1254vm-test-run-nixbot-gitlab> gitlab # [  166.603853] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.426Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1255vm-test-run-nixbot-gitlab> gitlab # [  166.606390] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.428Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1256vm-test-run-nixbot-gitlab> gitlab # [  166.608748] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.431Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1257vm-test-run-nixbot-gitlab> gitlab # [  166.611075] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.432Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1258vm-test-run-nixbot-gitlab> gitlab # [  166.613491] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.435Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1259vm-test-run-nixbot-gitlab> gitlab # [  166.615816] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.437Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1260vm-test-run-nixbot-gitlab> gitlab # [  166.618248] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.439Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1261vm-test-run-nixbot-gitlab> gitlab # [  166.620718] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.441Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1262vm-test-run-nixbot-gitlab> gitlab # [  166.623139] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.443Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1263vm-test-run-nixbot-gitlab> gitlab # [  166.625240] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.446Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1264vm-test-run-nixbot-gitlab> gitlab # [  166.627563] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.449Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1265vm-test-run-nixbot-gitlab> gitlab # [  166.631079] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.451Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1266vm-test-run-nixbot-gitlab> gitlab # [  166.633493] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.452Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1267vm-test-run-nixbot-gitlab> gitlab # [  166.635773] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.463Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1268vm-test-run-nixbot-gitlab> gitlab # [  166.638208] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.465Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1269vm-test-run-nixbot-gitlab> gitlab # [  166.640488] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.467Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1270vm-test-run-nixbot-gitlab> gitlab # [  166.642779] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.470Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1271vm-test-run-nixbot-gitlab> gitlab # [  166.644936] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.473Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1272vm-test-run-nixbot-gitlab> gitlab # [  166.647157] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.475Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1273vm-test-run-nixbot-gitlab> gitlab # [  166.649698] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.478Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1274vm-test-run-nixbot-gitlab> gitlab # [  166.652073] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.480Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1275vm-test-run-nixbot-gitlab> gitlab # [  166.654228] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.482Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1276vm-test-run-nixbot-gitlab> gitlab # [  166.656570] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.484Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1277vm-test-run-nixbot-gitlab> gitlab # [  166.658813] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:52:33.486Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1278vm-test-run-nixbot-gitlab> gitlab # [  170.246224] gitlab-workhorse[1302]: time="2026-07-04T10:52:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1279vm-test-run-nixbot-gitlab> gitlab # [  180.247322] gitlab-workhorse[1302]: time="2026-07-04T10:52:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1280vm-test-run-nixbot-gitlab> gitlab # [  190.258108] gitlab-workhorse[1302]: time="2026-07-04T10:52:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1281vm-test-run-nixbot-gitlab> gitlab # [  200.268947] gitlab-workhorse[1302]: time="2026-07-04T10:53:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1282vm-test-run-nixbot-gitlab> gitlab # [  202.269800] sidekiq-cluster[2431]: InitializerConnections Query: SET client_min_messages TO 'warning'1283vm-test-run-nixbot-gitlab> gitlab # [  202.273884] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1284vm-test-run-nixbot-gitlab> gitlab # [  202.279254] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1285vm-test-run-nixbot-gitlab> gitlab # [  202.283368] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1286vm-test-run-nixbot-gitlab> gitlab # [  202.287691] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1287vm-test-run-nixbot-gitlab> gitlab # [  202.291618] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1288vm-test-run-nixbot-gitlab> gitlab # [  202.296391] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1289vm-test-run-nixbot-gitlab> gitlab # [  202.301826] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1290vm-test-run-nixbot-gitlab> gitlab # [  202.311445] sidekiq-cluster[2431]: InitializerConnections Query: SET standard_conforming_strings = on1291vm-test-run-nixbot-gitlab> gitlab # [  202.315573] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1292vm-test-run-nixbot-gitlab> gitlab # [  202.320264] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1293vm-test-run-nixbot-gitlab> gitlab # [  202.324169] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1294vm-test-run-nixbot-gitlab> gitlab # [  202.327981] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1295vm-test-run-nixbot-gitlab> gitlab # [  202.331208] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1296vm-test-run-nixbot-gitlab> gitlab # [  202.334405] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1297vm-test-run-nixbot-gitlab> gitlab # [  202.338108] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1298vm-test-run-nixbot-gitlab> gitlab # [  202.366770] sidekiq-cluster[2431]: InitializerConnections Query: SET intervalstyle = iso_86011299vm-test-run-nixbot-gitlab> gitlab # [  202.370813] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1300vm-test-run-nixbot-gitlab> gitlab # [  202.375483] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1301vm-test-run-nixbot-gitlab> gitlab # [  202.379489] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1302vm-test-run-nixbot-gitlab> gitlab # [  202.383784] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1303vm-test-run-nixbot-gitlab> gitlab # [  202.387684] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1304vm-test-run-nixbot-gitlab> gitlab # [  202.391400] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1305vm-test-run-nixbot-gitlab> gitlab # [  202.395614] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1306vm-test-run-nixbot-gitlab> gitlab # [  202.399698] sidekiq-cluster[2431]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1307vm-test-run-nixbot-gitlab> gitlab # [  202.404000] sidekiq-cluster[2431]: FROM pg_type as t1308vm-test-run-nixbot-gitlab> gitlab # [  202.405568] sidekiq-cluster[2431]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1309vm-test-run-nixbot-gitlab> gitlab # [  202.409252] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1310vm-test-run-nixbot-gitlab> gitlab # [  202.412056] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1311vm-test-run-nixbot-gitlab> gitlab # [  202.414421] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1312vm-test-run-nixbot-gitlab> gitlab # [  202.416937] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1313vm-test-run-nixbot-gitlab> gitlab # [  202.419058] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1314vm-test-run-nixbot-gitlab> gitlab # [  202.421290] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1315vm-test-run-nixbot-gitlab> gitlab # [  202.423825] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1316vm-test-run-nixbot-gitlab> gitlab # [  202.426546] sidekiq-cluster[2431]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1317vm-test-run-nixbot-gitlab> gitlab # [  202.428353] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1318vm-test-run-nixbot-gitlab> gitlab # [  202.430551] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1319vm-test-run-nixbot-gitlab> gitlab # [  202.432429] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1320vm-test-run-nixbot-gitlab> gitlab # [  202.434340] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1321vm-test-run-nixbot-gitlab> gitlab # [  202.436025] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1322vm-test-run-nixbot-gitlab> gitlab # [  202.437702] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1323vm-test-run-nixbot-gitlab> gitlab # [  202.439738] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1324vm-test-run-nixbot-gitlab> gitlab # [  203.339713] sidekiq-cluster[2431]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware'1325vm-test-run-nixbot-gitlab> gitlab # [  203.340083] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:341:in `block in current_feature_value'1326vm-test-run-nixbot-gitlab> gitlab # [  203.340195] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:356:in `with_feature'1327vm-test-run-nixbot-gitlab> gitlab # [  203.340290] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:337:in `current_feature_value'1328vm-test-run-nixbot-gitlab> gitlab # [  203.340384] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:130:in `enabled?'1329vm-test-run-nixbot-gitlab> gitlab # [  203.340480] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/feature.rb:145:in `disabled?'1330vm-test-run-nixbot-gitlab> gitlab # [  203.340575] sidekiq-cluster[2431]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1331vm-test-run-nixbot-gitlab> gitlab # [  203.340670] sidekiq-cluster[2431]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1332vm-test-run-nixbot-gitlab> gitlab # [  204.108541] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:10.940Z","message":"Booted Rails 7.2.3 application in production environment"}1333vm-test-run-nixbot-gitlab> gitlab # [  204.119432] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:10.951Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1334vm-test-run-nixbot-gitlab> gitlab # [  204.121674] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:10.951Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1335vm-test-run-nixbot-gitlab> gitlab # [  204.123431] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:10.952Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1336vm-test-run-nixbot-gitlab> gitlab # [  204.126174] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:10.957Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1337vm-test-run-nixbot-gitlab> gitlab # [  206.747915] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:13.578Z","message":"Cleaning working queues"}1338vm-test-run-nixbot-gitlab> gitlab # [  206.752730] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:13.579Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b03a2613c1d64ac71e686d17","created_at":"2026-07-04T10:51:42.329Z","trace_propagation_headers":{"sentry-trace":"c3a4d3c2109145e5b28a0480de167be8-147dda835faf41d1","baggage":"sentry-trace_id=c3a4d3c2109145e5b28a0480de167be8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"66f7dad1eecdbcfc2d8741cd267e647f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T10:51:42.340Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b03a2613c1d64ac71e686d17: start","job_status":"start","queue_duration_s":91.238934,"scheduling_latency_s":91.238995}1339vm-test-run-nixbot-gitlab> gitlab # [  207.016687] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:13.847Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b03a2613c1d64ac71e686d17","created_at":"2026-07-04T10:51:42.329Z","trace_propagation_headers":{"sentry-trace":"c3a4d3c2109145e5b28a0480de167be8-147dda835faf41d1","baggage":"sentry-trace_id=c3a4d3c2109145e5b28a0480de167be8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"66f7dad1eecdbcfc2d8741cd267e647f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T10:51:42.340Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b03a2613c1d64ac71e686d17: done: 0.267631 sec","job_status":"done","queue_duration_s":91.238934,"scheduling_latency_s":91.238995,"redis_calls":29,"redis_duration_s":0.008917,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.005079,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002313,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1448196,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.267631,"completed_at":"2026-07-04T10:53:13.847Z","load_balancing_strategy":"primary","db_duration_s":0.105152,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1340vm-test-run-nixbot-gitlab> gitlab # [  207.462699] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.293Z","pid":2383,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1341vm-test-run-nixbot-gitlab> gitlab # [  207.468633] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.293Z","pid":2383,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1342vm-test-run-nixbot-gitlab> gitlab # [  207.473881] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.293Z","pid":2383,"message":"! #\u003cThread:0x0000fafb43705640 /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/09wikq0z4z0s8mkafl8f2b6yp0p78frd-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1343vm-test-run-nixbot-gitlab> gitlab # [  207.489233] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.293Z","pid":2383,"message":"! #\u003cThread:0x0000fafb4395f328 /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/q2bwpwxril8z19gq9b1nnv72drlwhhg8-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1344vm-test-run-nixbot-gitlab> gitlab # [  207.504606] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.293Z","pid":2383,"message":"Use Ctrl-C to stop"}1345vm-test-run-nixbot-gitlab> gitlab # [  207.508513] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.294Z","pid":2383,"message":"Sending status to systemd every 1.0 sec"}1346vm-test-run-nixbot-gitlab> gitlab # [  207.978832] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.810Z","pid":2383,"message":"- Worker 0 (PID: 3203) booted in 0.46s, phase: 0"}1347vm-test-run-nixbot-gitlab> gitlab # [  208.065084] bundle[2383]: {"timestamp":"2026-07-04T10:53:14.896Z","pid":2383,"message":"- Worker 1 (PID: 3207) booted in 0.51s, phase: 0"}1348vm-test-run-nixbot-gitlab> gitlab # [  208.071552] systemd[1]: Started gitlab.service.1349vm-test-run-nixbot-gitlab> gitlab # [  208.073950] systemd[1]: Reached target Common target for all GitLab services..1350vm-test-run-nixbot-gitlab> gitlab # [  208.079798] systemd[1]: Starting setup-gitlab.service...1351vm-test-run-nixbot-gitlab> gitlab # [  208.130843] setup-gitlab-start[3240]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1352vm-test-run-nixbot-gitlab> gitlab # [  211.733050] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:53:18 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 35571353vm-test-run-nixbot-gitlab> gitlab # [  211.741207] setup-gitlab-start[3268]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token1354vm-test-run-nixbot-gitlab> gitlab # [  211.743804] setup-gitlab-start[3269]: ++ jq -r .access_token1355vm-test-run-nixbot-gitlab> gitlab # [  213.349952] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:53:20 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 15771356vm-test-run-nixbot-gitlab> gitlab # [  213.353323] setup-gitlab-start[3240]: + BEARER=ffd715455da63ed7f3a75aedcd2b2fe85d92c167afb2196e09179b85c0f8343d1357vm-test-run-nixbot-gitlab> gitlab # [  213.356462] setup-gitlab-start[3240]: + AUTH='Authorization: Bearer ffd715455da63ed7f3a75aedcd2b2fe85d92c167afb2196e09179b85c0f8343d'1358vm-test-run-nixbot-gitlab> gitlab # [  213.359293] setup-gitlab-start[3240]: + curl -fs -X PUT -H 'Authorization: Bearer ffd715455da63ed7f3a75aedcd2b2fe85d92c167afb2196e09179b85c0f8343d' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true'1359vm-test-run-nixbot-gitlab> gitlab # [  214.297593] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:53:21 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 9171360vm-test-run-nixbot-gitlab> gitlab # [  214.299392] setup-gitlab-start[3240]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear1361vm-test-run-nixbot-gitlab> gitlab # [  214.318067] runuser[3291]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1362vm-test-run-nixbot-gitlab> gitlab # [  214.829367] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1363vm-test-run-nixbot-gitlab> gitlab # [  214.835976] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1364vm-test-run-nixbot-gitlab> gitlab # [  214.842381] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1365vm-test-run-nixbot-gitlab> gitlab # [  215.577780] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1366vm-test-run-nixbot-gitlab> gitlab # [  215.578103] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1367vm-test-run-nixbot-gitlab> gitlab # [  215.582955] setup-gitlab-start[3292]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1368vm-test-run-nixbot-gitlab> gitlab # [  218.171026] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:25.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"dcafe02b37d175bd97fd601a","created_at":"2026-07-04T10:53:24.950Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9804103fb9e3da2371e0dcb039cb4741","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:24.997Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-dcafe02b37d175bd97fd601a: start","job_status":"start","queue_duration_s":0.00301,"scheduling_latency_s":0.003062}1369vm-test-run-nixbot-gitlab> gitlab # [  218.428884] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:25.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"dcafe02b37d175bd97fd601a","created_at":"2026-07-04T10:53:24.950Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9804103fb9e3da2371e0dcb039cb4741","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:24.997Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-dcafe02b37d175bd97fd601a: done: 0.258281 sec","job_status":"done","queue_duration_s":0.00301,"scheduling_latency_s":0.003062,"redis_calls":10,"redis_duration_s":0.012410000000000001,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00176,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00822914,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.258281,"completed_at":"2026-07-04T10:53:25.259Z","load_balancing_strategy":"primary","db_duration_s":0.096666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370vm-test-run-nixbot-gitlab> gitlab # [  218.509241] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:25.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a6e82ab239802829cb061ae3","created_at":"2026-07-04T10:53:25.010Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.051Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a6e82ab239802829cb061ae3: start","job_status":"start","queue_duration_s":0.267305,"scheduling_latency_s":0.267356}1371vm-test-run-nixbot-gitlab> gitlab # [  219.172181] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a6e82ab239802829cb061ae3","created_at":"2026-07-04T10:53:25.010Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.051Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a6e82ab239802829cb061ae3: done: 0.682427 sec","job_status":"done","queue_duration_s":0.267305,"scheduling_latency_s":0.267356,"redis_calls":71,"redis_duration_s":0.220797,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.013821,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00297,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.184658,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.019348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.074,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.074,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.273738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0031706,"duration_s":0.682427,"completed_at":"2026-07-04T10:53:26.001Z","load_balancing_strategy":"primary","db_duration_s":0.074318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372vm-test-run-nixbot-gitlab> gitlab # [  219.261869] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"30c49e474e6de152bc81d6f1","created_at":"2026-07-04T10:53:25.099Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a55eb98e5bea76a5f13d85adb9f494d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.108Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-30c49e474e6de152bc81d6f1: start","job_status":"start","queue_duration_s":0.899384,"scheduling_latency_s":0.899434}1373vm-test-run-nixbot-gitlab> gitlab # [  220.004764] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"30c49e474e6de152bc81d6f1","created_at":"2026-07-04T10:53:25.099Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a55eb98e5bea76a5f13d85adb9f494d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.108Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-30c49e474e6de152bc81d6f1: done: 0.827766 sec","job_status":"done","queue_duration_s":0.899384,"scheduling_latency_s":0.899434,"redis_calls":14,"redis_duration_s":0.008237000000000001,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002445,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003572,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.056,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.58911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00363676,"exclusive_lock_hold_duration_s":0.74275302,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.827766,"completed_at":"2026-07-04T10:53:26.835Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374vm-test-run-nixbot-gitlab> gitlab # [  220.103053] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"dbcf7bc982e8cdd173538502","created_at":"2026-07-04T10:53:25.113Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c176a7246b588d304f51c0126eb9bed2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.125Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-dbcf7bc982e8cdd173538502: start","job_status":"start","queue_duration_s":1.712721,"scheduling_latency_s":1.712747}1375vm-test-run-nixbot-gitlab> gitlab # [  220.121179] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"dbcf7bc982e8cdd173538502","created_at":"2026-07-04T10:53:25.113Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c176a7246b588d304f51c0126eb9bed2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.125Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-dbcf7bc982e8cdd173538502: done: 0.024386 sec","job_status":"done","queue_duration_s":1.712721,"scheduling_latency_s":1.712747,"redis_calls":12,"redis_duration_s":0.003022,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001046,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034246,"duration_s":0.024386,"completed_at":"2026-07-04T10:53:26.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376vm-test-run-nixbot-gitlab> gitlab # [  220.166418] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ce3a25e96776b4ce1309270b","created_at":"2026-07-04T10:53:25.138Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff6127d45fa575b758ecf931359a044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.148Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ce3a25e96776b4ce1309270b: start","job_status":"start","queue_duration_s":1.715748,"scheduling_latency_s":1.715769}1377vm-test-run-nixbot-gitlab> gitlab # [  220.181271] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.876Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1378vm-test-run-nixbot-gitlab> gitlab # [  220.184531] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ce3a25e96776b4ce1309270b","created_at":"2026-07-04T10:53:25.138Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff6127d45fa575b758ecf931359a044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.148Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ce3a25e96776b4ce1309270b: done: 0.019982 sec","job_status":"done","queue_duration_s":1.715748,"scheduling_latency_s":1.715769,"redis_calls":10,"redis_duration_s":0.002446,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048364,"duration_s":0.019982,"completed_at":"2026-07-04T10:53:26.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00053,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379vm-test-run-nixbot-gitlab> gitlab # [  220.227409] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.885Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"bab428c8f6643a98f47a6fe8","created_at":"2026-07-04T10:53:25.177Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b5735eac724fcf9f832f1826a8b2ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T10:53:25.191Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-bab428c8f6643a98f47a6fe8: start","job_status":"start","queue_duration_s":1.693477,"scheduling_latency_s":1.693497}1380vm-test-run-nixbot-gitlab> gitlab # [  220.241264] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.897Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1381vm-test-run-nixbot-gitlab> gitlab # [  220.245043] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"bab428c8f6643a98f47a6fe8","created_at":"2026-07-04T10:53:25.177Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5b5735eac724fcf9f832f1826a8b2ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T10:53:25.191Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-bab428c8f6643a98f47a6fe8: done: 0.019867 sec","job_status":"done","queue_duration_s":1.693477,"scheduling_latency_s":1.693497,"redis_calls":10,"redis_duration_s":0.002477,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003224,"duration_s":0.019867,"completed_at":"2026-07-04T10:53:26.905Z","load_balancing_strategy":"primary","db_duration_s":0.000545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382vm-test-run-nixbot-gitlab> gitlab # [  220.286839] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8103f074d6ed27903ac7ce80","created_at":"2026-07-04T10:53:25.225Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5252d7b62a6c943c60dc09d3727f1a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.255Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8103f074d6ed27903ac7ce80: start","job_status":"start","queue_duration_s":1.652285,"scheduling_latency_s":1.652304}1383vm-test-run-nixbot-gitlab> gitlab # [  220.301804] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"8103f074d6ed27903ac7ce80","created_at":"2026-07-04T10:53:25.225Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5252d7b62a6c943c60dc09d3727f1a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.255Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-8103f074d6ed27903ac7ce80: done: 0.049615 sec","job_status":"done","queue_duration_s":1.652285,"scheduling_latency_s":1.652304,"redis_calls":12,"redis_duration_s":0.003151,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00056252,"exclusive_lock_hold_duration_s":0.02836458,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.049615,"completed_at":"2026-07-04T10:53:26.956Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384vm-test-run-nixbot-gitlab> gitlab # [  220.347105] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6fb2d96b65e04bbb280ea43b","created_at":"2026-07-04T10:53:25.343Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc12f5e53d7f8d241b87bde2cc34e266","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T10:53:25.376Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6fb2d96b65e04bbb280ea43b: start","job_status":"start","queue_duration_s":1.581861,"scheduling_latency_s":1.581882}1385vm-test-run-nixbot-gitlab> gitlab # [  220.361604] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6fb2d96b65e04bbb280ea43b","created_at":"2026-07-04T10:53:25.343Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc12f5e53d7f8d241b87bde2cc34e266","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T10:53:25.376Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6fb2d96b65e04bbb280ea43b: done: 0.038766 sec","job_status":"done","queue_duration_s":1.581861,"scheduling_latency_s":1.581882,"redis_calls":10,"redis_duration_s":0.002496,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035968,"duration_s":0.038766,"completed_at":"2026-07-04T10:53:26.997Z","load_balancing_strategy":"primary","db_duration_s":0.009117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386vm-test-run-nixbot-gitlab> gitlab # [  220.402899] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:26.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7693110128a741bf703a2765","created_at":"2026-07-04T10:53:25.388Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18a231bec4454b6f78eed1c8335f27a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.417Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7693110128a741bf703a2765: start","job_status":"start","queue_duration_s":1.581778,"scheduling_latency_s":1.581797}1387vm-test-run-nixbot-gitlab> gitlab # [  220.417962] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.011Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1388vm-test-run-nixbot-gitlab> gitlab # [  220.421447] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7693110128a741bf703a2765","created_at":"2026-07-04T10:53:25.388Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18a231bec4454b6f78eed1c8335f27a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.417Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7693110128a741bf703a2765: done: 0.020152 sec","job_status":"done","queue_duration_s":1.581778,"scheduling_latency_s":1.581797,"redis_calls":10,"redis_duration_s":0.002378,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033218,"duration_s":0.020152,"completed_at":"2026-07-04T10:53:27.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389vm-test-run-nixbot-gitlab> gitlab # [  220.464352] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"165310f63711d6ffc0240600","created_at":"2026-07-04T10:53:25.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ad0b4b8b96cc47fe33700a42279748e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.531Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-165310f63711d6ffc0240600: start","job_status":"start","queue_duration_s":1.488818,"scheduling_latency_s":1.488837}1390vm-test-run-nixbot-gitlab> gitlab # [  220.479343] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.093Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"165310f63711d6ffc0240600","created_at":"2026-07-04T10:53:25.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ad0b4b8b96cc47fe33700a42279748e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.531Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-165310f63711d6ffc0240600: done: 0.072699 sec","job_status":"done","queue_duration_s":1.488818,"scheduling_latency_s":1.488837,"redis_calls":12,"redis_duration_s":0.0039000000000000003,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000485,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074546,"exclusive_lock_hold_duration_s":0.02650188,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.072699,"completed_at":"2026-07-04T10:53:27.093Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391vm-test-run-nixbot-gitlab> gitlab # [  220.525596] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.095Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4d70769c91c453b70b4786ac","created_at":"2026-07-04T10:53:25.569Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f8519cf39718c136128bd4f95e02f6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.590Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4d70769c91c453b70b4786ac: start","job_status":"start","queue_duration_s":1.504574,"scheduling_latency_s":1.504597}1392vm-test-run-nixbot-gitlab> gitlab # [  220.540610] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4d70769c91c453b70b4786ac","created_at":"2026-07-04T10:53:25.569Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f8519cf39718c136128bd4f95e02f6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.590Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4d70769c91c453b70b4786ac: done: 0.024387 sec","job_status":"done","queue_duration_s":1.504574,"scheduling_latency_s":1.504597,"redis_calls":10,"redis_duration_s":0.002763,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001495,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034792,"duration_s":0.024387,"completed_at":"2026-07-04T10:53:27.119Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393vm-test-run-nixbot-gitlab> gitlab # [  220.586891] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.121Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3ba7ef49f2c0dcaa5e3fefc6","created_at":"2026-07-04T10:53:25.615Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f204c497e0d8497901cd9a216d82fd89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T10:53:25.644Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3ba7ef49f2c0dcaa5e3fefc6: start","job_status":"start","queue_duration_s":1.476787,"scheduling_latency_s":1.476807}1394vm-test-run-nixbot-gitlab> gitlab # [  220.600817] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.135Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1395vm-test-run-nixbot-gitlab> gitlab # [  220.604204] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3ba7ef49f2c0dcaa5e3fefc6","created_at":"2026-07-04T10:53:25.615Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f204c497e0d8497901cd9a216d82fd89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T10:53:25.644Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3ba7ef49f2c0dcaa5e3fefc6: done: 0.021389 sec","job_status":"done","queue_duration_s":1.476787,"scheduling_latency_s":1.476807,"redis_calls":10,"redis_duration_s":0.002512,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001529,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035316,"duration_s":0.021389,"completed_at":"2026-07-04T10:53:27.142Z","load_balancing_strategy":"primary","db_duration_s":0.000648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396vm-test-run-nixbot-gitlab> gitlab # [  220.646279] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0bd1c7994521644bda42d3d3","created_at":"2026-07-04T10:53:25.675Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23f0ce24dc0973aaa9fd26f2bfff3a5a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.705Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0bd1c7994521644bda42d3d3: start","job_status":"start","queue_duration_s":1.438988,"scheduling_latency_s":1.439007}1397vm-test-run-nixbot-gitlab> gitlab # [  220.660821] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0bd1c7994521644bda42d3d3","created_at":"2026-07-04T10:53:25.675Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23f0ce24dc0973aaa9fd26f2bfff3a5a","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.705Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0bd1c7994521644bda42d3d3: done: 0.030166 sec","job_status":"done","queue_duration_s":1.438988,"scheduling_latency_s":1.439007,"redis_calls":12,"redis_duration_s":0.003216,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000542,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058844,"exclusive_lock_hold_duration_s":0.00807598,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.030166,"completed_at":"2026-07-04T10:53:27.174Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398vm-test-run-nixbot-gitlab> gitlab # [  220.704805] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6be29edbb0d421b40c37d08c","created_at":"2026-07-04T10:53:25.727Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d36e0c88f75aac7cae30f50564608ae","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.757Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6be29edbb0d421b40c37d08c: start","job_status":"start","queue_duration_s":1.41868,"scheduling_latency_s":1.418696}1399vm-test-run-nixbot-gitlab> gitlab # [  220.719600] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6be29edbb0d421b40c37d08c","created_at":"2026-07-04T10:53:25.727Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d36e0c88f75aac7cae30f50564608ae","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.757Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6be29edbb0d421b40c37d08c: done: 0.036773 sec","job_status":"done","queue_duration_s":1.41868,"scheduling_latency_s":1.418696,"redis_calls":13,"redis_duration_s":0.0032790000000000002,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01419074,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036773,"completed_at":"2026-07-04T10:53:27.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007011,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1400vm-test-run-nixbot-gitlab> gitlab # [  220.766291] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"4f5ed2af82a81d119811f9d2","created_at":"2026-07-04T10:53:25.781Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b3a3667c413233230f3003dbb8ad8dd9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.806Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-4f5ed2af82a81d119811f9d2: start","job_status":"start","queue_duration_s":1.408118,"scheduling_latency_s":1.408136}1401vm-test-run-nixbot-gitlab> gitlab # [  220.779793] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"4f5ed2af82a81d119811f9d2","created_at":"2026-07-04T10:53:25.781Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b3a3667c413233230f3003dbb8ad8dd9","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.806Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-4f5ed2af82a81d119811f9d2: done: 0.040059 sec","job_status":"done","queue_duration_s":1.408118,"scheduling_latency_s":1.408136,"redis_calls":12,"redis_duration_s":0.004048,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000687,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001458,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064868,"exclusive_lock_hold_duration_s":0.01223168,"duration_s":0.040059,"completed_at":"2026-07-04T10:53:27.254Z","load_balancing_strategy":"primary","db_duration_s":0.006819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402vm-test-run-nixbot-gitlab> gitlab # [  220.821748] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4bc00f2b158a5bf87991555b","created_at":"2026-07-04T10:53:25.863Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f7b3e5a622e92064bed6a01dbbee71b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.887Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4bc00f2b158a5bf87991555b: start","job_status":"start","queue_duration_s":1.368933,"scheduling_latency_s":1.368953}1403vm-test-run-nixbot-gitlab> gitlab # [  220.835633] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"4bc00f2b158a5bf87991555b","created_at":"2026-07-04T10:53:25.863Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f7b3e5a622e92064bed6a01dbbee71b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.887Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-4bc00f2b158a5bf87991555b: done: 0.031008 sec","job_status":"done","queue_duration_s":1.368933,"scheduling_latency_s":1.368953,"redis_calls":12,"redis_duration_s":0.003314,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057026,"exclusive_lock_hold_duration_s":0.00923256,"duration_s":0.031008,"completed_at":"2026-07-04T10:53:27.287Z","load_balancing_strategy":"primary","db_duration_s":0.005603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404vm-test-run-nixbot-gitlab> gitlab # [  220.878034] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"93fce40a389608e5c0119375","created_at":"2026-07-04T10:53:25.912Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8290a7bf5498c93c84f98b35d0555b50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.937Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-93fce40a389608e5c0119375: start","job_status":"start","queue_duration_s":1.351361,"scheduling_latency_s":1.351379}1405vm-test-run-nixbot-gitlab> gitlab # [  220.892451] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"93fce40a389608e5c0119375","created_at":"2026-07-04T10:53:25.912Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8290a7bf5498c93c84f98b35d0555b50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.937Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-93fce40a389608e5c0119375: done: 0.026107 sec","job_status":"done","queue_duration_s":1.351361,"scheduling_latency_s":1.351379,"redis_calls":10,"redis_duration_s":0.001595,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00029,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002217,"duration_s":0.026107,"completed_at":"2026-07-04T10:53:27.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406vm-test-run-nixbot-gitlab> gitlab # [  220.936676] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"836763bca324dac1f76527d7","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-836763bca324dac1f76527d7: start","job_status":"start","queue_duration_s":1.356847,"scheduling_latency_s":1.356866}1407vm-test-run-nixbot-gitlab> gitlab # [  220.949589] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"836763bca324dac1f76527d7","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-836763bca324dac1f76527d7: done: 0.028904 sec","job_status":"done","queue_duration_s":1.356847,"scheduling_latency_s":1.356866,"redis_calls":16,"redis_duration_s":0.0026,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000305,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001583,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028448,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028904,"completed_at":"2026-07-04T10:53:27.345Z","load_balancing_strategy":"primary","db_duration_s":0.000964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408vm-test-run-nixbot-gitlab> gitlab # [  220.993522] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee91b29230f5698f3562569","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee91b29230f5698f3562569: start","job_status":"start","queue_duration_s":1.386942,"scheduling_latency_s":1.386961}1409vm-test-run-nixbot-gitlab> gitlab # [  221.006419] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee91b29230f5698f3562569","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee91b29230f5698f3562569: done: 0.016305 sec","job_status":"done","queue_duration_s":1.386942,"scheduling_latency_s":1.386961,"redis_calls":13,"redis_duration_s":0.0016549999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016305,"completed_at":"2026-07-04T10:53:27.362Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410vm-test-run-nixbot-gitlab> gitlab # [  221.046836] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cb6702b4905f954550ed46e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cb6702b4905f954550ed46e: start","job_status":"start","queue_duration_s":1.405614,"scheduling_latency_s":1.405634}1411vm-test-run-nixbot-gitlab> gitlab # [  221.059767] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cb6702b4905f954550ed46e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cb6702b4905f954550ed46e: done: 0.015839 sec","job_status":"done","queue_duration_s":1.405614,"scheduling_latency_s":1.405634,"redis_calls":13,"redis_duration_s":0.001649,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015839,"completed_at":"2026-07-04T10:53:27.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412vm-test-run-nixbot-gitlab> gitlab # [  221.100472] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a00b8b343e7a375a358d015a","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a00b8b343e7a375a358d015a: start","job_status":"start","queue_duration_s":1.422707,"scheduling_latency_s":1.422726}1413vm-test-run-nixbot-gitlab> gitlab # [  221.113114] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a00b8b343e7a375a358d015a","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a00b8b343e7a375a358d015a: done: 0.01552 sec","job_status":"done","queue_duration_s":1.422707,"scheduling_latency_s":1.422726,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01552,"completed_at":"2026-07-04T10:53:27.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414vm-test-run-nixbot-gitlab> gitlab # [  221.153222] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e90fb1b0d4be30a3909de","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e90fb1b0d4be30a3909de: start","job_status":"start","queue_duration_s":1.439405,"scheduling_latency_s":1.439423}1415vm-test-run-nixbot-gitlab> gitlab # [  221.166916] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e90fb1b0d4be30a3909de","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e90fb1b0d4be30a3909de: done: 0.015423 sec","job_status":"done","queue_duration_s":1.439405,"scheduling_latency_s":1.439423,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015423,"completed_at":"2026-07-04T10:53:27.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416vm-test-run-nixbot-gitlab> gitlab # [  221.206955] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94b3adf4f7f41c426e7658c2","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94b3adf4f7f41c426e7658c2: start","job_status":"start","queue_duration_s":1.456335,"scheduling_latency_s":1.456356}1417vm-test-run-nixbot-gitlab> gitlab # [  221.219740] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94b3adf4f7f41c426e7658c2","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94b3adf4f7f41c426e7658c2: done: 0.017224 sec","job_status":"done","queue_duration_s":1.456335,"scheduling_latency_s":1.456356,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017224,"completed_at":"2026-07-04T10:53:27.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418vm-test-run-nixbot-gitlab> gitlab # [  221.259489] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"113a32798f3966945a3bf296","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-113a32798f3966945a3bf296: start","job_status":"start","queue_duration_s":1.474713,"scheduling_latency_s":1.474732}1419vm-test-run-nixbot-gitlab> gitlab # [  221.271943] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"113a32798f3966945a3bf296","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-113a32798f3966945a3bf296: done: 0.01592 sec","job_status":"done","queue_duration_s":1.474713,"scheduling_latency_s":1.474732,"redis_calls":13,"redis_duration_s":0.001628,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01592,"completed_at":"2026-07-04T10:53:27.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420vm-test-run-nixbot-gitlab> gitlab # [  221.311687] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6545fae9e7a0b60b356c0994","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6545fae9e7a0b60b356c0994: start","job_status":"start","queue_duration_s":1.491745,"scheduling_latency_s":1.491763}1421vm-test-run-nixbot-gitlab> gitlab # [  221.324834] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6545fae9e7a0b60b356c0994","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6545fae9e7a0b60b356c0994: done: 0.015956 sec","job_status":"done","queue_duration_s":1.491745,"scheduling_latency_s":1.491763,"redis_calls":13,"redis_duration_s":0.001769,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015956,"completed_at":"2026-07-04T10:53:27.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422vm-test-run-nixbot-gitlab> gitlab # [  221.364610] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b473a84e8d5478ba23e9ac","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b473a84e8d5478ba23e9ac: start","job_status":"start","queue_duration_s":1.508905,"scheduling_latency_s":1.508925}1423vm-test-run-nixbot-gitlab> gitlab # [  221.378603] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31b473a84e8d5478ba23e9ac","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31b473a84e8d5478ba23e9ac: done: 0.01659 sec","job_status":"done","queue_duration_s":1.508905,"scheduling_latency_s":1.508925,"redis_calls":13,"redis_duration_s":0.001823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01659,"completed_at":"2026-07-04T10:53:27.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424vm-test-run-nixbot-gitlab> gitlab # [  221.418355] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2031bdc2e9a5c6d24abe6db","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2031bdc2e9a5c6d24abe6db: start","job_status":"start","queue_duration_s":1.526685,"scheduling_latency_s":1.526705}1425vm-test-run-nixbot-gitlab> gitlab # [  221.430828] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2031bdc2e9a5c6d24abe6db","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2031bdc2e9a5c6d24abe6db: done: 0.016395 sec","job_status":"done","queue_duration_s":1.526685,"scheduling_latency_s":1.526705,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016395,"completed_at":"2026-07-04T10:53:27.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426vm-test-run-nixbot-gitlab> gitlab # [  221.470698] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"790c90107ba7356d09d21908","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-790c90107ba7356d09d21908: start","job_status":"start","queue_duration_s":1.544249,"scheduling_latency_s":1.544267}1427vm-test-run-nixbot-gitlab> gitlab # [  221.483401] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"790c90107ba7356d09d21908","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-790c90107ba7356d09d21908: done: 0.015538 sec","job_status":"done","queue_duration_s":1.544249,"scheduling_latency_s":1.544267,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015538,"completed_at":"2026-07-04T10:53:27.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428vm-test-run-nixbot-gitlab> gitlab # [  221.523358] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a7ac6f42f7abc2ac0f8f6f1","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a7ac6f42f7abc2ac0f8f6f1: start","job_status":"start","queue_duration_s":1.560904,"scheduling_latency_s":1.560923}1429vm-test-run-nixbot-gitlab> gitlab # [  221.536257] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a7ac6f42f7abc2ac0f8f6f1","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a7ac6f42f7abc2ac0f8f6f1: done: 0.016559 sec","job_status":"done","queue_duration_s":1.560904,"scheduling_latency_s":1.560923,"redis_calls":13,"redis_duration_s":0.001825,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016559,"completed_at":"2026-07-04T10:53:27.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430vm-test-run-nixbot-gitlab> gitlab # [  221.577024] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2e7a42f3ce9de2b9d1d7bd2","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2e7a42f3ce9de2b9d1d7bd2: start","job_status":"start","queue_duration_s":1.578849,"scheduling_latency_s":1.578871}1431vm-test-run-nixbot-gitlab> gitlab # [  221.589676] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2e7a42f3ce9de2b9d1d7bd2","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2e7a42f3ce9de2b9d1d7bd2: done: 0.016821 sec","job_status":"done","queue_duration_s":1.578849,"scheduling_latency_s":1.578871,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016821,"completed_at":"2026-07-04T10:53:27.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432vm-test-run-nixbot-gitlab> gitlab # [  221.629637] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"307a51cf77052394a666b474","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-307a51cf77052394a666b474: start","job_status":"start","queue_duration_s":1.597221,"scheduling_latency_s":1.597244}1433vm-test-run-nixbot-gitlab> gitlab # [  221.642136] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"307a51cf77052394a666b474","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-307a51cf77052394a666b474: done: 0.016849 sec","job_status":"done","queue_duration_s":1.597221,"scheduling_latency_s":1.597244,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016849,"completed_at":"2026-07-04T10:53:27.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434vm-test-run-nixbot-gitlab> gitlab # [  221.681891] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48cd4e416397da7ca0674f71","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48cd4e416397da7ca0674f71: start","job_status":"start","queue_duration_s":1.615435,"scheduling_latency_s":1.615458}1435vm-test-run-nixbot-gitlab> gitlab # [  221.694692] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48cd4e416397da7ca0674f71","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48cd4e416397da7ca0674f71: done: 0.016337 sec","job_status":"done","queue_duration_s":1.615435,"scheduling_latency_s":1.615458,"redis_calls":13,"redis_duration_s":0.001593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016337,"completed_at":"2026-07-04T10:53:27.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436vm-test-run-nixbot-gitlab> gitlab # [  221.734295] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12bb74a952a926cfaca934a0","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12bb74a952a926cfaca934a0: start","job_status":"start","queue_duration_s":1.633705,"scheduling_latency_s":1.633727}1437vm-test-run-nixbot-gitlab> gitlab # [  221.747000] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12bb74a952a926cfaca934a0","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12bb74a952a926cfaca934a0: done: 0.017018 sec","job_status":"done","queue_duration_s":1.633705,"scheduling_latency_s":1.633727,"redis_calls":13,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017018,"completed_at":"2026-07-04T10:53:27.610Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438vm-test-run-nixbot-gitlab> gitlab # [  221.788041] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2eab10b6bd68cef45350e63","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2eab10b6bd68cef45350e63: start","job_status":"start","queue_duration_s":1.652305,"scheduling_latency_s":1.652329}1439vm-test-run-nixbot-gitlab> gitlab # [  221.800442] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2eab10b6bd68cef45350e63","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2eab10b6bd68cef45350e63: done: 0.017231 sec","job_status":"done","queue_duration_s":1.652305,"scheduling_latency_s":1.652329,"redis_calls":13,"redis_duration_s":0.0019779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017231,"completed_at":"2026-07-04T10:53:27.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440vm-test-run-nixbot-gitlab> gitlab # [  221.840246] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efdc904d936b89ef660d0675","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efdc904d936b89ef660d0675: start","job_status":"start","queue_duration_s":1.670777,"scheduling_latency_s":1.670797}1441vm-test-run-nixbot-gitlab> gitlab # [  221.852962] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efdc904d936b89ef660d0675","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efdc904d936b89ef660d0675: done: 0.017112 sec","job_status":"done","queue_duration_s":1.670777,"scheduling_latency_s":1.670797,"redis_calls":13,"redis_duration_s":0.0017180000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017112,"completed_at":"2026-07-04T10:53:27.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442vm-test-run-nixbot-gitlab> gitlab # [  221.893566] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d1e975d6a446919097cd14","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d1e975d6a446919097cd14: start","job_status":"start","queue_duration_s":1.689095,"scheduling_latency_s":1.689114}1443vm-test-run-nixbot-gitlab> gitlab # [  221.906349] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d1e975d6a446919097cd14","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d1e975d6a446919097cd14: done: 0.016978 sec","job_status":"done","queue_duration_s":1.689095,"scheduling_latency_s":1.689114,"redis_calls":13,"redis_duration_s":0.001729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016978,"completed_at":"2026-07-04T10:53:27.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444vm-test-run-nixbot-gitlab> gitlab # [  221.946135] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6eb198fb3347477d31f476a","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6eb198fb3347477d31f476a: start","job_status":"start","queue_duration_s":1.70726,"scheduling_latency_s":1.707278}1445vm-test-run-nixbot-gitlab> gitlab # [  221.959179] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6eb198fb3347477d31f476a","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6eb198fb3347477d31f476a: done: 0.016645 sec","job_status":"done","queue_duration_s":1.70726,"scheduling_latency_s":1.707278,"redis_calls":13,"redis_duration_s":0.002015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016645,"completed_at":"2026-07-04T10:53:27.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446vm-test-run-nixbot-gitlab> gitlab # [  222.000674] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"575ccf6609110cd23b1bc5ba","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-575ccf6609110cd23b1bc5ba: start","job_status":"start","queue_duration_s":1.725094,"scheduling_latency_s":1.725114}1447vm-test-run-nixbot-gitlab> gitlab # [  222.013709] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"575ccf6609110cd23b1bc5ba","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-575ccf6609110cd23b1bc5ba: done: 0.016442 sec","job_status":"done","queue_duration_s":1.725094,"scheduling_latency_s":1.725114,"redis_calls":13,"redis_duration_s":0.001719,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016442,"completed_at":"2026-07-04T10:53:27.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448vm-test-run-nixbot-gitlab> gitlab # [  222.053483] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11686556e53f467a50ed5c3f","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11686556e53f467a50ed5c3f: start","job_status":"start","queue_duration_s":1.742719,"scheduling_latency_s":1.742738}1449vm-test-run-nixbot-gitlab> gitlab # [  222.066269] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11686556e53f467a50ed5c3f","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11686556e53f467a50ed5c3f: done: 0.016441 sec","job_status":"done","queue_duration_s":1.742719,"scheduling_latency_s":1.742738,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016441,"completed_at":"2026-07-04T10:53:27.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450vm-test-run-nixbot-gitlab> gitlab # [  222.106851] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68daf9e7f1c6eacde8dd0025","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68daf9e7f1c6eacde8dd0025: start","job_status":"start","queue_duration_s":1.760585,"scheduling_latency_s":1.760606}1451vm-test-run-nixbot-gitlab> gitlab # [  222.119854] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68daf9e7f1c6eacde8dd0025","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68daf9e7f1c6eacde8dd0025: done: 0.016837 sec","job_status":"done","queue_duration_s":1.760585,"scheduling_latency_s":1.760606,"redis_calls":13,"redis_duration_s":0.002054,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016837,"completed_at":"2026-07-04T10:53:27.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452vm-test-run-nixbot-gitlab> gitlab # [  222.160378] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c116b4484f11ca1041993ed8","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c116b4484f11ca1041993ed8: start","job_status":"start","queue_duration_s":1.778918,"scheduling_latency_s":1.778938}1453vm-test-run-nixbot-gitlab> gitlab # [  222.173072] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c116b4484f11ca1041993ed8","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c116b4484f11ca1041993ed8: done: 0.016229 sec","job_status":"done","queue_duration_s":1.778918,"scheduling_latency_s":1.778938,"redis_calls":13,"redis_duration_s":0.001749,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016229,"completed_at":"2026-07-04T10:53:27.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454vm-test-run-nixbot-gitlab> gitlab # [  222.214656] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b303f669aeefe8e8be55ec9c","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b303f669aeefe8e8be55ec9c: start","job_status":"start","queue_duration_s":1.796653,"scheduling_latency_s":1.796672}1455vm-test-run-nixbot-gitlab> gitlab # [  222.227473] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b303f669aeefe8e8be55ec9c","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b303f669aeefe8e8be55ec9c: done: 0.01707 sec","job_status":"done","queue_duration_s":1.796653,"scheduling_latency_s":1.796672,"redis_calls":13,"redis_duration_s":0.001726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01707,"completed_at":"2026-07-04T10:53:27.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456vm-test-run-nixbot-gitlab> gitlab # [  222.267621] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8295c3968e402b7e3c0ad613","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8295c3968e402b7e3c0ad613: start","job_status":"start","queue_duration_s":1.815021,"scheduling_latency_s":1.815041}1457vm-test-run-nixbot-gitlab> gitlab # [  222.280217] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8295c3968e402b7e3c0ad613","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8295c3968e402b7e3c0ad613: done: 0.015232 sec","job_status":"done","queue_duration_s":1.815021,"scheduling_latency_s":1.815041,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015232,"completed_at":"2026-07-04T10:53:27.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458vm-test-run-nixbot-gitlab> gitlab # [  222.320144] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f72799f0f0faa9b5163affa","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f72799f0f0faa9b5163affa: start","job_status":"start","queue_duration_s":1.831548,"scheduling_latency_s":1.831567}1459vm-test-run-nixbot-gitlab> gitlab # [  222.332790] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f72799f0f0faa9b5163affa","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f72799f0f0faa9b5163affa: done: 0.014899 sec","job_status":"done","queue_duration_s":1.831548,"scheduling_latency_s":1.831567,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014899,"completed_at":"2026-07-04T10:53:27.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460vm-test-run-nixbot-gitlab> gitlab # [  222.372706] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16893aaa342a7495fc74dc35","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16893aaa342a7495fc74dc35: start","job_status":"start","queue_duration_s":1.847519,"scheduling_latency_s":1.847536}1461vm-test-run-nixbot-gitlab> gitlab # [  222.385132] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16893aaa342a7495fc74dc35","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16893aaa342a7495fc74dc35: done: 0.014878 sec","job_status":"done","queue_duration_s":1.847519,"scheduling_latency_s":1.847536,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014878,"completed_at":"2026-07-04T10:53:27.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462vm-test-run-nixbot-gitlab> gitlab # [  222.427247] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3745202757284768bbf4770","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3745202757284768bbf4770: start","job_status":"start","queue_duration_s":1.863602,"scheduling_latency_s":1.86362}1463vm-test-run-nixbot-gitlab> gitlab # [  222.465404] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3745202757284768bbf4770","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3745202757284768bbf4770: done: 0.015315 sec","job_status":"done","queue_duration_s":1.863602,"scheduling_latency_s":1.86362,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015315,"completed_at":"2026-07-04T10:53:27.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464vm-test-run-nixbot-gitlab> gitlab # [  222.535722] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f2444b12cee143450bda420","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f2444b12cee143450bda420: start","job_status":"start","queue_duration_s":1.879981,"scheduling_latency_s":1.88}1465vm-test-run-nixbot-gitlab> gitlab # [  222.550041] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f2444b12cee143450bda420","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f2444b12cee143450bda420: done: 0.015098 sec","job_status":"done","queue_duration_s":1.879981,"scheduling_latency_s":1.88,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015098,"completed_at":"2026-07-04T10:53:27.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466vm-test-run-nixbot-gitlab> gitlab # [  222.590358] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c0d949516907c2cf3a3634e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c0d949516907c2cf3a3634e: start","job_status":"start","queue_duration_s":1.896174,"scheduling_latency_s":1.896192}1467vm-test-run-nixbot-gitlab> gitlab # [  222.604667] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c0d949516907c2cf3a3634e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c0d949516907c2cf3a3634e: done: 0.016092 sec","job_status":"done","queue_duration_s":1.896174,"scheduling_latency_s":1.896192,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016092,"completed_at":"2026-07-04T10:53:27.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468vm-test-run-nixbot-gitlab> gitlab # [  222.648416] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880ddc51554a94f4a002a36e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880ddc51554a94f4a002a36e: start","job_status":"start","queue_duration_s":1.913464,"scheduling_latency_s":1.913483}1469vm-test-run-nixbot-gitlab> gitlab # [  222.662535] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880ddc51554a94f4a002a36e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880ddc51554a94f4a002a36e: done: 0.017659 sec","job_status":"done","queue_duration_s":1.913464,"scheduling_latency_s":1.913483,"redis_calls":13,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017659,"completed_at":"2026-07-04T10:53:27.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470vm-test-run-nixbot-gitlab> gitlab # [  222.702504] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:27.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e2fbaec3d222cbe68e4c9ae","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e2fbaec3d222cbe68e4c9ae: start","job_status":"start","queue_duration_s":1.932676,"scheduling_latency_s":1.932698}1471vm-test-run-nixbot-gitlab> gitlab # [  222.715257] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e2fbaec3d222cbe68e4c9ae","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e2fbaec3d222cbe68e4c9ae: done: 0.137084 sec","job_status":"done","queue_duration_s":1.932676,"scheduling_latency_s":1.932698,"redis_calls":13,"redis_duration_s":0.0022519999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.137084,"completed_at":"2026-07-04T10:53:28.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472vm-test-run-nixbot-gitlab> gitlab # [  222.755230] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12a8368930f1e3b0e10b25a3","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12a8368930f1e3b0e10b25a3: start","job_status":"start","queue_duration_s":2.074107,"scheduling_latency_s":2.074153}1473vm-test-run-nixbot-gitlab> gitlab # [  222.767757] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12a8368930f1e3b0e10b25a3","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12a8368930f1e3b0e10b25a3: done: 0.03798 sec","job_status":"done","queue_duration_s":2.074107,"scheduling_latency_s":2.074153,"redis_calls":13,"redis_duration_s":0.004191,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002781,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03798,"completed_at":"2026-07-04T10:53:28.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474vm-test-run-nixbot-gitlab> gitlab # [  222.808529] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14247a573bc3cd5a487f8b34","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14247a573bc3cd5a487f8b34: start","job_status":"start","queue_duration_s":2.11471,"scheduling_latency_s":2.114741}1475vm-test-run-nixbot-gitlab> gitlab # [  222.821135] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14247a573bc3cd5a487f8b34","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14247a573bc3cd5a487f8b34: done: 0.026207 sec","job_status":"done","queue_duration_s":2.11471,"scheduling_latency_s":2.114741,"redis_calls":13,"redis_duration_s":0.003561,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002428,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026207,"completed_at":"2026-07-04T10:53:28.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476vm-test-run-nixbot-gitlab> gitlab # [  222.860864] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89737fb3146c2e46779b1057","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89737fb3146c2e46779b1057: start","job_status":"start","queue_duration_s":2.143051,"scheduling_latency_s":2.143076}1477vm-test-run-nixbot-gitlab> gitlab # [  222.873414] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89737fb3146c2e46779b1057","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89737fb3146c2e46779b1057: done: 0.023816 sec","job_status":"done","queue_duration_s":2.143051,"scheduling_latency_s":2.143076,"redis_calls":13,"redis_duration_s":0.003336,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023816,"completed_at":"2026-07-04T10:53:28.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478vm-test-run-nixbot-gitlab> gitlab # [  222.913150] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3e0533229a52e09c1a1e61f","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3e0533229a52e09c1a1e61f: start","job_status":"start","queue_duration_s":2.1689,"scheduling_latency_s":2.168923}1479vm-test-run-nixbot-gitlab> gitlab # [  222.925718] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3e0533229a52e09c1a1e61f","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3e0533229a52e09c1a1e61f: done: 0.02104 sec","job_status":"done","queue_duration_s":2.1689,"scheduling_latency_s":2.168923,"redis_calls":13,"redis_duration_s":0.0034929999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02104,"completed_at":"2026-07-04T10:53:28.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480vm-test-run-nixbot-gitlab> gitlab # [  222.965399] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23322ca2d3ee49bdd1d12e41","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23322ca2d3ee49bdd1d12e41: start","job_status":"start","queue_duration_s":2.191651,"scheduling_latency_s":2.191672}1481vm-test-run-nixbot-gitlab> gitlab # [  222.978192] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23322ca2d3ee49bdd1d12e41","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23322ca2d3ee49bdd1d12e41: done: 0.020045 sec","job_status":"done","queue_duration_s":2.191651,"scheduling_latency_s":2.191672,"redis_calls":13,"redis_duration_s":0.003194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020045,"completed_at":"2026-07-04T10:53:28.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482vm-test-run-nixbot-gitlab> gitlab # [  223.018638] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90eab0be1f12302565312049","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90eab0be1f12302565312049: start","job_status":"start","queue_duration_s":2.213145,"scheduling_latency_s":2.213166}1483vm-test-run-nixbot-gitlab> gitlab # [  223.031532] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90eab0be1f12302565312049","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90eab0be1f12302565312049: done: 0.018434 sec","job_status":"done","queue_duration_s":2.213145,"scheduling_latency_s":2.213166,"redis_calls":13,"redis_duration_s":0.00278,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018434,"completed_at":"2026-07-04T10:53:28.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484vm-test-run-nixbot-gitlab> gitlab # [  223.071629] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa65d64a71ea44a58539e92","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa65d64a71ea44a58539e92: start","job_status":"start","queue_duration_s":2.232994,"scheduling_latency_s":2.233013}1485vm-test-run-nixbot-gitlab> gitlab # [  223.084327] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa65d64a71ea44a58539e92","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa65d64a71ea44a58539e92: done: 0.018162 sec","job_status":"done","queue_duration_s":2.232994,"scheduling_latency_s":2.233013,"redis_calls":13,"redis_duration_s":0.0029709999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018162,"completed_at":"2026-07-04T10:53:28.210Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486vm-test-run-nixbot-gitlab> gitlab # [  223.124858] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84948542cd0f96b287ade9e9","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84948542cd0f96b287ade9e9: start","job_status":"start","queue_duration_s":2.252508,"scheduling_latency_s":2.252527}1487vm-test-run-nixbot-gitlab> gitlab # [  223.137493] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84948542cd0f96b287ade9e9","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84948542cd0f96b287ade9e9: done: 0.016122 sec","job_status":"done","queue_duration_s":2.252508,"scheduling_latency_s":2.252527,"redis_calls":13,"redis_duration_s":0.002648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016122,"completed_at":"2026-07-04T10:53:28.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488vm-test-run-nixbot-gitlab> gitlab # [  223.177338] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200c49657fbe363e350b7042","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200c49657fbe363e350b7042: start","job_status":"start","queue_duration_s":2.270133,"scheduling_latency_s":2.27015}1489vm-test-run-nixbot-gitlab> gitlab # [  223.189748] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"200c49657fbe363e350b7042","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-200c49657fbe363e350b7042: done: 0.016271 sec","job_status":"done","queue_duration_s":2.270133,"scheduling_latency_s":2.27015,"redis_calls":13,"redis_duration_s":0.002911,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016271,"completed_at":"2026-07-04T10:53:28.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490vm-test-run-nixbot-gitlab> gitlab # [  223.230781] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daa2f319b0eed0063ed6b9de","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daa2f319b0eed0063ed6b9de: start","job_status":"start","queue_duration_s":2.287912,"scheduling_latency_s":2.287929}1491vm-test-run-nixbot-gitlab> gitlab # [  223.243625] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daa2f319b0eed0063ed6b9de","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daa2f319b0eed0063ed6b9de: done: 0.015663 sec","job_status":"done","queue_duration_s":2.287912,"scheduling_latency_s":2.287929,"redis_calls":13,"redis_duration_s":0.0025930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001672,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015663,"completed_at":"2026-07-04T10:53:28.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492vm-test-run-nixbot-gitlab> gitlab # [  223.283720] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25ff501d17b0a52a02aeb7cc","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25ff501d17b0a52a02aeb7cc: start","job_status":"start","queue_duration_s":2.304908,"scheduling_latency_s":2.304925}1493vm-test-run-nixbot-gitlab> gitlab # [  223.296690] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25ff501d17b0a52a02aeb7cc","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25ff501d17b0a52a02aeb7cc: done: 0.015908 sec","job_status":"done","queue_duration_s":2.304908,"scheduling_latency_s":2.304925,"redis_calls":13,"redis_duration_s":0.0027930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015908,"completed_at":"2026-07-04T10:53:28.280Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494vm-test-run-nixbot-gitlab> gitlab # [  223.336664] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8a8cb105cfed08752a96bb","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8a8cb105cfed08752a96bb: start","job_status":"start","queue_duration_s":2.322452,"scheduling_latency_s":2.32247}1495vm-test-run-nixbot-gitlab> gitlab # [  223.349239] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8a8cb105cfed08752a96bb","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8a8cb105cfed08752a96bb: done: 0.015947 sec","job_status":"done","queue_duration_s":2.322452,"scheduling_latency_s":2.32247,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015947,"completed_at":"2026-07-04T10:53:28.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496vm-test-run-nixbot-gitlab> gitlab # [  223.389030] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47bb58c4eade9337ccfaa8bc","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47bb58c4eade9337ccfaa8bc: start","job_status":"start","queue_duration_s":2.339926,"scheduling_latency_s":2.339943}1497vm-test-run-nixbot-gitlab> gitlab # [  223.401807] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47bb58c4eade9337ccfaa8bc","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47bb58c4eade9337ccfaa8bc: done: 0.015526 sec","job_status":"done","queue_duration_s":2.339926,"scheduling_latency_s":2.339943,"redis_calls":13,"redis_duration_s":0.002445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015526,"completed_at":"2026-07-04T10:53:28.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498vm-test-run-nixbot-gitlab> gitlab # [  223.443181] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a02835e4555947b794c9eb6","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a02835e4555947b794c9eb6: start","job_status":"start","queue_duration_s":2.356896,"scheduling_latency_s":2.356914}1499vm-test-run-nixbot-gitlab> gitlab # [  223.455850] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a02835e4555947b794c9eb6","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a02835e4555947b794c9eb6: done: 0.015232 sec","job_status":"done","queue_duration_s":2.356896,"scheduling_latency_s":2.356914,"redis_calls":13,"redis_duration_s":0.001759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015232,"completed_at":"2026-07-04T10:53:28.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500vm-test-run-nixbot-gitlab> gitlab # [  223.496072] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c7c6a78c591ab293fca6c7e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c7c6a78c591ab293fca6c7e: start","job_status":"start","queue_duration_s":2.373301,"scheduling_latency_s":2.37332}1501vm-test-run-nixbot-gitlab> gitlab # [  223.508530] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c7c6a78c591ab293fca6c7e","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c7c6a78c591ab293fca6c7e: done: 0.015536 sec","job_status":"done","queue_duration_s":2.373301,"scheduling_latency_s":2.37332,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015536,"completed_at":"2026-07-04T10:53:28.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502vm-test-run-nixbot-gitlab> gitlab # [  223.548641] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc506f82ea06eda0f777bbc0","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc506f82ea06eda0f777bbc0: start","job_status":"start","queue_duration_s":2.389957,"scheduling_latency_s":2.389974}1503vm-test-run-nixbot-gitlab> gitlab # [  223.561106] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc506f82ea06eda0f777bbc0","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc506f82ea06eda0f777bbc0: done: 0.015081 sec","job_status":"done","queue_duration_s":2.389957,"scheduling_latency_s":2.389974,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015081,"completed_at":"2026-07-04T10:53:28.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504vm-test-run-nixbot-gitlab> gitlab # [  223.600887] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e243c67a78ea19a53f353359","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e243c67a78ea19a53f353359: start","job_status":"start","queue_duration_s":2.40618,"scheduling_latency_s":2.406196}1505vm-test-run-nixbot-gitlab> gitlab # [  223.614373] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e243c67a78ea19a53f353359","created_at":"2026-07-04T10:53:25.426Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-7c39cece78084e60","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"458d0b1fa52c109516519fff097bdc3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.959Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e243c67a78ea19a53f353359: done: 0.01664 sec","job_status":"done","queue_duration_s":2.40618,"scheduling_latency_s":2.406196,"redis_calls":13,"redis_duration_s":0.0018050000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01664,"completed_at":"2026-07-04T10:53:28.382Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506vm-test-run-nixbot-gitlab> gitlab # [  223.654607] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5b45faa84ad3f79e63697e61","created_at":"2026-07-04T10:53:25.963Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f925e0f8951ba3781b2bc3449723d4d2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.994Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5b45faa84ad3f79e63697e61: start","job_status":"start","queue_duration_s":2.389104,"scheduling_latency_s":2.389121}1507vm-test-run-nixbot-gitlab> gitlab # [  223.669238] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.433Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5b45faa84ad3f79e63697e61","created_at":"2026-07-04T10:53:25.963Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f925e0f8951ba3781b2bc3449723d4d2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:25.994Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5b45faa84ad3f79e63697e61: done: 0.050032 sec","job_status":"done","queue_duration_s":2.389104,"scheduling_latency_s":2.389121,"redis_calls":16,"redis_duration_s":0.004269,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003214,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002444,"duration_s":0.050032,"completed_at":"2026-07-04T10:53:28.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508vm-test-run-nixbot-gitlab> gitlab # [  223.712176] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"627976511bdf69e4928880cf","created_at":"2026-07-04T10:53:26.009Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc66e616badfa17d2a5db3f32b838b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.024Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-627976511bdf69e4928880cf: start","job_status":"start","queue_duration_s":2.410208,"scheduling_latency_s":2.410225}1509vm-test-run-nixbot-gitlab> gitlab # [  223.726762] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.447Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1510vm-test-run-nixbot-gitlab> gitlab # [  223.730778] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"627976511bdf69e4928880cf","created_at":"2026-07-04T10:53:26.009Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cfc66e616badfa17d2a5db3f32b838b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.024Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-627976511bdf69e4928880cf: done: 0.019984 sec","job_status":"done","queue_duration_s":2.410208,"scheduling_latency_s":2.410225,"redis_calls":10,"redis_duration_s":0.001524,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023774,"duration_s":0.019984,"completed_at":"2026-07-04T10:53:28.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511vm-test-run-nixbot-gitlab> gitlab # [  223.774531] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"cd1847055fa0cdaf0bc7a093","created_at":"2026-07-04T10:53:26.077Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02da22960d675a766f6435a5103b9a67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.130Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-cd1847055fa0cdaf0bc7a093: start","job_status":"start","queue_duration_s":2.326011,"scheduling_latency_s":2.326029}1512vm-test-run-nixbot-gitlab> gitlab # [  223.788855] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"cd1847055fa0cdaf0bc7a093","created_at":"2026-07-04T10:53:26.077Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02da22960d675a766f6435a5103b9a67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.130Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-cd1847055fa0cdaf0bc7a093: done: 0.066502 sec","job_status":"done","queue_duration_s":2.326011,"scheduling_latency_s":2.326029,"redis_calls":20,"redis_duration_s":0.00266,"redis_read_bytes":19,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001094,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000241,"redis_queues_read_bytes":2,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022738,"duration_s":0.066502,"completed_at":"2026-07-04T10:53:28.522Z","load_balancing_strategy":"primary","db_duration_s":0.019381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513vm-test-run-nixbot-gitlab> gitlab # [  223.833113] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"68efef73c802498196b2ac87","created_at":"2026-07-04T10:53:26.166Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.180Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-68efef73c802498196b2ac87: start","job_status":"start","queue_duration_s":2.343559,"scheduling_latency_s":2.343577}1514vm-test-run-nixbot-gitlab> gitlab # [  223.847339] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"68efef73c802498196b2ac87","created_at":"2026-07-04T10:53:26.166Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.180Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-68efef73c802498196b2ac87: done: 0.036046 sec","job_status":"done","queue_duration_s":2.343559,"scheduling_latency_s":2.343577,"redis_calls":19,"redis_duration_s":0.002671,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000582,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000197,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022088,"duration_s":0.036046,"completed_at":"2026-07-04T10:53:28.559Z","load_balancing_strategy":"primary","db_duration_s":0.003223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515vm-test-run-nixbot-gitlab> gitlab # [  223.890692] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a8ff196253fd16b3bcd8be90","created_at":"2026-07-04T10:53:26.189Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161c3a5eb43bf60fb3df1f2c71a90df4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.202Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a8ff196253fd16b3bcd8be90: start","job_status":"start","queue_duration_s":2.358466,"scheduling_latency_s":2.358484}1516vm-test-run-nixbot-gitlab> gitlab # [  223.905092] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a8ff196253fd16b3bcd8be90","created_at":"2026-07-04T10:53:26.189Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161c3a5eb43bf60fb3df1f2c71a90df4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.202Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a8ff196253fd16b3bcd8be90: done: 0.320607 sec","job_status":"done","queue_duration_s":2.358466,"scheduling_latency_s":2.358484,"redis_calls":574,"redis_duration_s":0.058964,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.057909,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.295739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023448,"duration_s":0.320607,"completed_at":"2026-07-04T10:53:28.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517vm-test-run-nixbot-gitlab> gitlab # [  223.947870] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d2266e2401d73d71d77ca46","created_at":"2026-07-04T10:53:26.212Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9282c2dd3ccdc6e70e993e9e25cc050","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.227Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d2266e2401d73d71d77ca46: start","job_status":"start","queue_duration_s":2.655623,"scheduling_latency_s":2.655642}1518vm-test-run-nixbot-gitlab> gitlab # [  223.962601] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.896Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1519vm-test-run-nixbot-gitlab> gitlab # [  223.966384] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8d2266e2401d73d71d77ca46","created_at":"2026-07-04T10:53:26.212Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9282c2dd3ccdc6e70e993e9e25cc050","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T10:53:26.227Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8d2266e2401d73d71d77ca46: done: 0.020248 sec","job_status":"done","queue_duration_s":2.655623,"scheduling_latency_s":2.655642,"redis_calls":10,"redis_duration_s":0.001431,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000331,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023576,"duration_s":0.020248,"completed_at":"2026-07-04T10:53:28.903Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520vm-test-run-nixbot-gitlab> gitlab # [  224.009419] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"4850e9b7494d2458c9892d9e","created_at":"2026-07-04T10:53:28.474Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4ed2aa195150464e","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"02da22960d675a766f6435a5103b9a67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.515Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-4850e9b7494d2458c9892d9e: start","job_status":"start","queue_duration_s":0.389422,"scheduling_latency_s":0.389441}1521vm-test-run-nixbot-gitlab> gitlab # [  224.023331] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"4850e9b7494d2458c9892d9e","created_at":"2026-07-04T10:53:28.474Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4ed2aa195150464e","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"02da22960d675a766f6435a5103b9a67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.515Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-4850e9b7494d2458c9892d9e: done: 0.034887 sec","job_status":"done","queue_duration_s":0.389422,"scheduling_latency_s":0.389441,"redis_calls":13,"redis_duration_s":0.001809,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022596,"duration_s":0.034887,"completed_at":"2026-07-04T10:53:28.939Z","load_balancing_strategy":"primary","db_duration_s":0.007377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522vm-test-run-nixbot-gitlab> gitlab # [  224.064825] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.940Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5967360b8708843e2a893eb5","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5967360b8708843e2a893eb5: start","job_status":"start","queue_duration_s":0.387924,"scheduling_latency_s":0.387943}1523vm-test-run-nixbot-gitlab> gitlab # [  224.077372] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.957Z","message":"No existing merge request to be cleaned up."}1524vm-test-run-nixbot-gitlab> gitlab # [  224.079000] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.970Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5967360b8708843e2a893eb5","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5967360b8708843e2a893eb5: done: 0.029935 sec","job_status":"done","queue_duration_s":0.387924,"scheduling_latency_s":0.387943,"redis_calls":13,"redis_duration_s":0.0017310000000000001,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022934,"duration_s":0.029935,"completed_at":"2026-07-04T10:53:28.970Z","load_balancing_strategy":"primary","db_duration_s":0.005326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525vm-test-run-nixbot-gitlab> gitlab # [  224.120442] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.972Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"707a129704db257e5d5e872f","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-707a129704db257e5d5e872f: start","job_status":"start","queue_duration_s":0.419191,"scheduling_latency_s":0.41921}1526vm-test-run-nixbot-gitlab> gitlab # [  224.132846] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.984Z","message":"No existing merge request to be cleaned up."}1527vm-test-run-nixbot-gitlab> gitlab # [  224.134628] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.992Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"707a129704db257e5d5e872f","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-707a129704db257e5d5e872f: done: 0.020753 sec","job_status":"done","queue_duration_s":0.419191,"scheduling_latency_s":0.41921,"redis_calls":10,"redis_duration_s":0.001402,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000825,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020753,"completed_at":"2026-07-04T10:53:28.992Z","load_balancing_strategy":"primary","db_duration_s":0.002723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528vm-test-run-nixbot-gitlab> gitlab # [  224.173858] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:28.994Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"759753c283e4e6f7c5b3f7e7","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-759753c283e4e6f7c5b3f7e7: start","job_status":"start","queue_duration_s":0.441247,"scheduling_latency_s":0.441266}1529vm-test-run-nixbot-gitlab> gitlab # [  224.186545] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:29.005Z","message":"No existing merge request to be cleaned up."}1530vm-test-run-nixbot-gitlab> gitlab # [  224.188284] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:29.013Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"759753c283e4e6f7c5b3f7e7","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-759753c283e4e6f7c5b3f7e7: done: 0.019218 sec","job_status":"done","queue_duration_s":0.441247,"scheduling_latency_s":0.441266,"redis_calls":10,"redis_duration_s":0.001356,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019218,"completed_at":"2026-07-04T10:53:29.013Z","load_balancing_strategy":"primary","db_duration_s":0.002323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531vm-test-run-nixbot-gitlab> gitlab # [  224.227920] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:29.014Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d630351e70edb0fc27a8727f","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d630351e70edb0fc27a8727f: start","job_status":"start","queue_duration_s":0.461599,"scheduling_latency_s":0.461618}1532vm-test-run-nixbot-gitlab> gitlab # [  224.241175] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:29.025Z","message":"No existing merge request to be cleaned up."}1533vm-test-run-nixbot-gitlab> gitlab # [  224.242880] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:53:29.033Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d630351e70edb0fc27a8727f","created_at":"2026-07-04T10:53:28.540Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-fdc5ba13ce7145d1","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9d56979d62c98e1541f065bf3ca6144f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:53:28.552Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d630351e70edb0fc27a8727f: done: 0.019222 sec","job_status":"done","queue_duration_s":0.461599,"scheduling_latency_s":0.461618,"redis_calls":10,"redis_duration_s":0.001337,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019222,"completed_at":"2026-07-04T10:53:29.033Z","load_balancing_strategy":"primary","db_duration_s":0.002388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534vm-test-run-nixbot-gitlab> gitlab # [  254.251733] setup-gitlab-start[3292]: W, [2026-07-04T10:54:01.082260 #3292]  WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1535vm-test-run-nixbot-gitlab> gitlab # [  257.745027] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:04.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"942d66e47b0db45ffcf52ec7","created_at":"2026-07-04T10:54:04.547Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"207fcb602a4a5c71f65e2a62cc819a6e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.567Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-942d66e47b0db45ffcf52ec7: start","job_status":"start","queue_duration_s":0.007398,"scheduling_latency_s":0.007458}1536vm-test-run-nixbot-gitlab> gitlab # [  258.282933] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:05.113Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"942d66e47b0db45ffcf52ec7","created_at":"2026-07-04T10:54:04.547Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"207fcb602a4a5c71f65e2a62cc819a6e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.567Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-942d66e47b0db45ffcf52ec7: done: 0.538957 sec","job_status":"done","queue_duration_s":0.007398,"scheduling_latency_s":0.007458,"redis_calls":7,"redis_duration_s":0.022018000000000003,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.016604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005414,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.265,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.265,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.538957,"completed_at":"2026-07-04T10:54:05.113Z","load_balancing_strategy":"primary","db_duration_s":0.409322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537vm-test-run-nixbot-gitlab> gitlab # [  258.464054] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:05.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8580d7e41f957ef9a547cc2d","created_at":"2026-07-04T10:54:04.627Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.638Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8580d7e41f957ef9a547cc2d: start","job_status":"start","queue_duration_s":0.656314,"scheduling_latency_s":0.656376}1538vm-test-run-nixbot-gitlab> gitlab # [  259.226228] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.055Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8580d7e41f957ef9a547cc2d","created_at":"2026-07-04T10:54:04.627Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.638Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8580d7e41f957ef9a547cc2d: done: 0.760793 sec","job_status":"done","queue_duration_s":0.656314,"scheduling_latency_s":0.656376,"redis_calls":60,"redis_duration_s":0.13670100000000002,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.001299,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.128173,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.007229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.240985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.760793,"completed_at":"2026-07-04T10:54:06.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539vm-test-run-nixbot-gitlab> gitlab # [  259.229174] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9374052df8ceda1882a939eb","created_at":"2026-07-04T10:54:04.663Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f082e3b3a517d43b5882518e789f685","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.690Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9374052df8ceda1882a939eb: start","job_status":"start","queue_duration_s":1.369619,"scheduling_latency_s":1.369676}1540vm-test-run-nixbot-gitlab> gitlab # [  259.335615] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9374052df8ceda1882a939eb","created_at":"2026-07-04T10:54:04.663Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f082e3b3a517d43b5882518e789f685","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.690Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9374052df8ceda1882a939eb: done: 0.057204 sec","job_status":"done","queue_duration_s":1.369619,"scheduling_latency_s":1.369676,"redis_calls":12,"redis_duration_s":0.004517,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000742,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001396,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00122742,"exclusive_lock_hold_duration_s":0.01307778,"duration_s":0.057204,"completed_at":"2026-07-04T10:54:06.117Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541vm-test-run-nixbot-gitlab> gitlab # [  259.382589] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"53d572ab43d3c6cc59325c24","created_at":"2026-07-04T10:54:04.722Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc9fd08d305056f9e79fe962851b39d1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.776Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-53d572ab43d3c6cc59325c24: start","job_status":"start","queue_duration_s":1.342105,"scheduling_latency_s":1.342135}1542vm-test-run-nixbot-gitlab> gitlab # [  259.399428] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"53d572ab43d3c6cc59325c24","created_at":"2026-07-04T10:54:04.722Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc9fd08d305056f9e79fe962851b39d1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.776Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-53d572ab43d3c6cc59325c24: done: 0.03343 sec","job_status":"done","queue_duration_s":1.342105,"scheduling_latency_s":1.342135,"redis_calls":10,"redis_duration_s":0.001847,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029104,"duration_s":0.03343,"completed_at":"2026-07-04T10:54:06.152Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543vm-test-run-nixbot-gitlab> gitlab # [  259.444703] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.153Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"19930d964864590b07ae5c04","created_at":"2026-07-04T10:54:04.826Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11cf9076ee595fda9136b53101459b65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.836Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-19930d964864590b07ae5c04: start","job_status":"start","queue_duration_s":1.317244,"scheduling_latency_s":1.317268}1544vm-test-run-nixbot-gitlab> gitlab # [  259.553668] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"19930d964864590b07ae5c04","created_at":"2026-07-04T10:54:04.826Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11cf9076ee595fda9136b53101459b65","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.836Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-19930d964864590b07ae5c04: done: 0.230823 sec","job_status":"done","queue_duration_s":1.317244,"scheduling_latency_s":1.317268,"redis_calls":11,"redis_duration_s":0.002706,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001391,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.191772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051938,"exclusive_lock_hold_duration_s":0.21112934,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.230823,"completed_at":"2026-07-04T10:54:06.384Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545vm-test-run-nixbot-gitlab> gitlab # [  259.639506] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2e9bf61db593d962e3309952","created_at":"2026-07-04T10:54:04.858Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a67e57e571f53be407389839e6471909","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.870Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2e9bf61db593d962e3309952: start","job_status":"start","queue_duration_s":1.515779,"scheduling_latency_s":1.515799}1546vm-test-run-nixbot-gitlab> gitlab # [  259.656457] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2e9bf61db593d962e3309952","created_at":"2026-07-04T10:54:04.858Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a67e57e571f53be407389839e6471909","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.870Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2e9bf61db593d962e3309952: done: 0.014944 sec","job_status":"done","queue_duration_s":1.515779,"scheduling_latency_s":1.515799,"redis_calls":8,"redis_duration_s":0.001136,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000124,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014944,"completed_at":"2026-07-04T10:54:06.400Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547vm-test-run-nixbot-gitlab> gitlab # [  259.701175] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bd43a72db66b92d3938e7ca3","created_at":"2026-07-04T10:54:04.898Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0e31fc44e1bd76c012c1856d234090","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.908Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bd43a72db66b92d3938e7ca3: start","job_status":"start","queue_duration_s":1.493662,"scheduling_latency_s":1.49368}1548vm-test-run-nixbot-gitlab> gitlab # [  259.715721] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.409Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1549vm-test-run-nixbot-gitlab> gitlab # [  259.719167] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bd43a72db66b92d3938e7ca3","created_at":"2026-07-04T10:54:04.898Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0e31fc44e1bd76c012c1856d234090","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:04.908Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bd43a72db66b92d3938e7ca3: done: 0.012853 sec","job_status":"done","queue_duration_s":1.493662,"scheduling_latency_s":1.49368,"redis_calls":7,"redis_duration_s":0.001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000295,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012853,"completed_at":"2026-07-04T10:54:06.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550vm-test-run-nixbot-gitlab> gitlab # [  259.760588] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fec89327f9f2059a175e3c2b","created_at":"2026-07-04T10:54:04.939Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f19c9d916b8d496a28ae892109bfe70","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T10:54:04.984Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fec89327f9f2059a175e3c2b: start","job_status":"start","queue_duration_s":1.432304,"scheduling_latency_s":1.432322}1551vm-test-run-nixbot-gitlab> gitlab # [  259.775283] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.423Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1552vm-test-run-nixbot-gitlab> gitlab # [  259.779212] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fec89327f9f2059a175e3c2b","created_at":"2026-07-04T10:54:04.939Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f19c9d916b8d496a28ae892109bfe70","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-04T10:54:04.984Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fec89327f9f2059a175e3c2b: done: 0.012576 sec","job_status":"done","queue_duration_s":1.432304,"scheduling_latency_s":1.432322,"redis_calls":7,"redis_duration_s":0.0009339999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012576,"completed_at":"2026-07-04T10:54:06.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553vm-test-run-nixbot-gitlab> gitlab # [  259.819779] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"03a2e9c845fe92c0416be9fd","created_at":"2026-07-04T10:54:05.021Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83eb0f19529953df4392542938d9ea98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.037Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-03a2e9c845fe92c0416be9fd: start","job_status":"start","queue_duration_s":1.392411,"scheduling_latency_s":1.392429}1554vm-test-run-nixbot-gitlab> gitlab # [  259.834927] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"03a2e9c845fe92c0416be9fd","created_at":"2026-07-04T10:54:05.021Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"83eb0f19529953df4392542938d9ea98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.037Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-03a2e9c845fe92c0416be9fd: done: 0.080239 sec","job_status":"done","queue_duration_s":1.392411,"scheduling_latency_s":1.392429,"redis_calls":9,"redis_duration_s":0.001552,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048414,"exclusive_lock_hold_duration_s":0.06443806,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.080239,"completed_at":"2026-07-04T10:54:06.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555vm-test-run-nixbot-gitlab> gitlab # [  259.879089] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"4448ae47f41b7a4af1d5695b","created_at":"2026-07-04T10:54:05.083Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"194e256a42f336ade7399ea8cfd737ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T10:54:05.093Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-4448ae47f41b7a4af1d5695b: start","job_status":"start","queue_duration_s":1.418152,"scheduling_latency_s":1.418175}1556vm-test-run-nixbot-gitlab> gitlab # [  259.892355] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"4448ae47f41b7a4af1d5695b","created_at":"2026-07-04T10:54:05.083Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"194e256a42f336ade7399ea8cfd737ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-04T10:54:05.093Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-4448ae47f41b7a4af1d5695b: done: 0.027495 sec","job_status":"done","queue_duration_s":1.418152,"scheduling_latency_s":1.418175,"redis_calls":7,"redis_duration_s":0.001249,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027495,"completed_at":"2026-07-04T10:54:06.539Z","load_balancing_strategy":"primary","db_duration_s":0.003817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557vm-test-run-nixbot-gitlab> gitlab # [  259.932100] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b63c3e88e4a2f646ccd84df","created_at":"2026-07-04T10:54:05.125Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902a62576527f2f74a44e8e8db09027c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.187Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b63c3e88e4a2f646ccd84df: start","job_status":"start","queue_duration_s":1.352856,"scheduling_latency_s":1.352878}1558vm-test-run-nixbot-gitlab> gitlab # [  259.946920] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.549Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1559vm-test-run-nixbot-gitlab> gitlab # [  259.950441] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b63c3e88e4a2f646ccd84df","created_at":"2026-07-04T10:54:05.125Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"902a62576527f2f74a44e8e8db09027c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.187Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b63c3e88e4a2f646ccd84df: done: 0.014988 sec","job_status":"done","queue_duration_s":1.352856,"scheduling_latency_s":1.352878,"redis_calls":7,"redis_duration_s":0.001117,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014988,"completed_at":"2026-07-04T10:54:06.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560vm-test-run-nixbot-gitlab> gitlab # [  259.992703] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a7a2a573f402fa9111ec1e97","created_at":"2026-07-04T10:54:05.258Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41290f3c07bc11bee1dc2c4256911b56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.273Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a7a2a573f402fa9111ec1e97: start","job_status":"start","queue_duration_s":1.283988,"scheduling_latency_s":1.284009}1561vm-test-run-nixbot-gitlab> gitlab # [  260.007621] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a7a2a573f402fa9111ec1e97","created_at":"2026-07-04T10:54:05.258Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41290f3c07bc11bee1dc2c4256911b56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.273Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a7a2a573f402fa9111ec1e97: done: 0.058894 sec","job_status":"done","queue_duration_s":1.283988,"scheduling_latency_s":1.284009,"redis_calls":9,"redis_duration_s":0.00182,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00062506,"exclusive_lock_hold_duration_s":0.02639038,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.058894,"completed_at":"2026-07-04T10:54:06.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562vm-test-run-nixbot-gitlab> gitlab # [  260.051357] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0eca46ed16df64fe3065833e","created_at":"2026-07-04T10:54:05.293Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a72858b55e0befc3c249cf286eed589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.375Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0eca46ed16df64fe3065833e: start","job_status":"start","queue_duration_s":1.241765,"scheduling_latency_s":1.241792}1563vm-test-run-nixbot-gitlab> gitlab # [  260.065996] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0eca46ed16df64fe3065833e","created_at":"2026-07-04T10:54:05.293Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a72858b55e0befc3c249cf286eed589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.375Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0eca46ed16df64fe3065833e: done: 0.017646 sec","job_status":"done","queue_duration_s":1.241765,"scheduling_latency_s":1.241792,"redis_calls":7,"redis_duration_s":0.001292,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017646,"completed_at":"2026-07-04T10:54:06.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564vm-test-run-nixbot-gitlab> gitlab # [  260.107559] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5922b8a061ec2bc749fcfd4a","created_at":"2026-07-04T10:54:05.395Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3b9722d72bb47bd86fc80eff1372e6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T10:54:05.401Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5922b8a061ec2bc749fcfd4a: start","job_status":"start","queue_duration_s":1.235719,"scheduling_latency_s":1.235741}1565vm-test-run-nixbot-gitlab> gitlab # [  260.121366] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.645Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1566vm-test-run-nixbot-gitlab> gitlab # [  260.124859] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5922b8a061ec2bc749fcfd4a","created_at":"2026-07-04T10:54:05.395Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3b9722d72bb47bd86fc80eff1372e6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-04T10:54:05.401Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5922b8a061ec2bc749fcfd4a: done: 0.014692 sec","job_status":"done","queue_duration_s":1.235719,"scheduling_latency_s":1.235741,"redis_calls":7,"redis_duration_s":0.001127,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014692,"completed_at":"2026-07-04T10:54:06.651Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567vm-test-run-nixbot-gitlab> gitlab # [  260.165566] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.652Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0c1c69ecc8415ba04444aeb7","created_at":"2026-07-04T10:54:05.422Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d70d5294d3d683a6ac4fdc18feb44e9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.446Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0c1c69ecc8415ba04444aeb7: start","job_status":"start","queue_duration_s":1.206217,"scheduling_latency_s":1.206237}1568vm-test-run-nixbot-gitlab> gitlab # [  260.180162] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.672Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0c1c69ecc8415ba04444aeb7","created_at":"2026-07-04T10:54:05.422Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d70d5294d3d683a6ac4fdc18feb44e9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.446Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0c1c69ecc8415ba04444aeb7: done: 0.019893 sec","job_status":"done","queue_duration_s":1.206217,"scheduling_latency_s":1.206237,"redis_calls":12,"redis_duration_s":0.001787,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000368,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026436,"duration_s":0.019893,"completed_at":"2026-07-04T10:54:06.672Z","load_balancing_strategy":"primary","db_duration_s":0.00061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569vm-test-run-nixbot-gitlab> gitlab # [  260.223606] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"14972ca0156b369e2788fdc9","created_at":"2026-07-04T10:54:05.489Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97d4e5f791f0c5daab57a22112bbbda6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.554Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-14972ca0156b369e2788fdc9: start","job_status":"start","queue_duration_s":1.119566,"scheduling_latency_s":1.119585}1570vm-test-run-nixbot-gitlab> gitlab # [  260.239190] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"14972ca0156b369e2788fdc9","created_at":"2026-07-04T10:54:05.489Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97d4e5f791f0c5daab57a22112bbbda6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.554Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-14972ca0156b369e2788fdc9: done: 0.019057 sec","job_status":"done","queue_duration_s":1.119566,"scheduling_latency_s":1.119585,"redis_calls":12,"redis_duration_s":0.001609,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025876,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.019057,"completed_at":"2026-07-04T10:54:06.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571vm-test-run-nixbot-gitlab> gitlab # [  260.285619] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0c5a1a9185972b443531d628","created_at":"2026-07-04T10:54:05.587Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03d20ec4545ca94773213911f29daf37","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.603Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0c5a1a9185972b443531d628: start","job_status":"start","queue_duration_s":1.090565,"scheduling_latency_s":1.090583}1572vm-test-run-nixbot-gitlab> gitlab # [  260.300222] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0c5a1a9185972b443531d628","created_at":"2026-07-04T10:54:05.587Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03d20ec4545ca94773213911f29daf37","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.603Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0c5a1a9185972b443531d628: done: 0.017041 sec","job_status":"done","queue_duration_s":1.090565,"scheduling_latency_s":1.090583,"redis_calls":9,"redis_duration_s":0.001398,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047246,"exclusive_lock_hold_duration_s":0.00328808,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017041,"completed_at":"2026-07-04T10:54:06.711Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1573vm-test-run-nixbot-gitlab> gitlab # [  260.343132] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"18138ee87537c09f0ed8522a","created_at":"2026-07-04T10:54:05.634Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c98dc8ee4f2ad19e0f41e5d59740d2d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.659Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-18138ee87537c09f0ed8522a: start","job_status":"start","queue_duration_s":1.053337,"scheduling_latency_s":1.053356}1574vm-test-run-nixbot-gitlab> gitlab # [  260.358361] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"18138ee87537c09f0ed8522a","created_at":"2026-07-04T10:54:05.634Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c98dc8ee4f2ad19e0f41e5d59740d2d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.659Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-18138ee87537c09f0ed8522a: done: 0.020694 sec","job_status":"done","queue_duration_s":1.053337,"scheduling_latency_s":1.053356,"redis_calls":10,"redis_duration_s":0.001442,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00550276,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.020694,"completed_at":"2026-07-04T10:54:06.733Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003462,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1575vm-test-run-nixbot-gitlab> gitlab # [  260.403143] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"664c162295b50d16721dae7c","created_at":"2026-07-04T10:54:05.725Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"697774cf67dff3896c2de924e298b1d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.738Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-664c162295b50d16721dae7c: start","job_status":"start","queue_duration_s":0.995719,"scheduling_latency_s":0.995737}1576vm-test-run-nixbot-gitlab> gitlab # [  260.417350] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"664c162295b50d16721dae7c","created_at":"2026-07-04T10:54:05.725Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"697774cf67dff3896c2de924e298b1d2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.738Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-664c162295b50d16721dae7c: done: 0.01654 sec","job_status":"done","queue_duration_s":0.995719,"scheduling_latency_s":0.995737,"redis_calls":9,"redis_duration_s":0.001308,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044916,"exclusive_lock_hold_duration_s":0.00336736,"duration_s":0.01654,"completed_at":"2026-07-04T10:54:06.750Z","load_balancing_strategy":"primary","db_duration_s":0.001627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1577vm-test-run-nixbot-gitlab> gitlab # [  260.458010] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.751Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"816b44a5499623675f010799","created_at":"2026-07-04T10:54:05.773Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46aaee74120a13a77e60d2d3ad72ff39","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.787Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-816b44a5499623675f010799: start","job_status":"start","queue_duration_s":0.964504,"scheduling_latency_s":0.964523}1578vm-test-run-nixbot-gitlab> gitlab # [  260.472777] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"816b44a5499623675f010799","created_at":"2026-07-04T10:54:05.773Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46aaee74120a13a77e60d2d3ad72ff39","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.787Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-816b44a5499623675f010799: done: 0.014662 sec","job_status":"done","queue_duration_s":0.964504,"scheduling_latency_s":0.964523,"redis_calls":7,"redis_duration_s":0.0009679999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014662,"completed_at":"2026-07-04T10:54:06.766Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1579vm-test-run-nixbot-gitlab> gitlab # [  260.514415] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.767Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0934691751288c6657eb27d4","created_at":"2026-07-04T10:54:05.811Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2782d01dfc0ac9414644463d9bc09757","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.821Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0934691751288c6657eb27d4: start","job_status":"start","queue_duration_s":0.946395,"scheduling_latency_s":0.946413}1580vm-test-run-nixbot-gitlab> gitlab # [  260.528743] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0934691751288c6657eb27d4","created_at":"2026-07-04T10:54:05.811Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2782d01dfc0ac9414644463d9bc09757","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.821Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0934691751288c6657eb27d4: done: 0.024457 sec","job_status":"done","queue_duration_s":0.946395,"scheduling_latency_s":0.946413,"redis_calls":7,"redis_duration_s":0.0010040000000000001,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024457,"completed_at":"2026-07-04T10:54:06.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1581vm-test-run-nixbot-gitlab> gitlab # [  260.571053] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"53fcbf0c10755469ada1760d","created_at":"2026-07-04T10:54:05.841Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be022165a3f71a7d6a0cacee8794f2b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.851Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-53fcbf0c10755469ada1760d: start","job_status":"start","queue_duration_s":0.942214,"scheduling_latency_s":0.942232}1582vm-test-run-nixbot-gitlab> gitlab # [  260.587160] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.869Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1583vm-test-run-nixbot-gitlab> gitlab # [  260.591224] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.875Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"53fcbf0c10755469ada1760d","created_at":"2026-07-04T10:54:05.841Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be022165a3f71a7d6a0cacee8794f2b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.851Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-53fcbf0c10755469ada1760d: done: 0.082711 sec","job_status":"done","queue_duration_s":0.942214,"scheduling_latency_s":0.942232,"redis_calls":7,"redis_duration_s":0.001066,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082711,"completed_at":"2026-07-04T10:54:06.875Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584vm-test-run-nixbot-gitlab> gitlab # [  260.632947] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6f3dd0dbaeba835661553e95","created_at":"2026-07-04T10:54:05.895Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.904Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6f3dd0dbaeba835661553e95: start","job_status":"start","queue_duration_s":0.972454,"scheduling_latency_s":0.972474}1585vm-test-run-nixbot-gitlab> gitlab # [  260.646744] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6f3dd0dbaeba835661553e95","created_at":"2026-07-04T10:54:05.895Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.904Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6f3dd0dbaeba835661553e95: done: 0.027093 sec","job_status":"done","queue_duration_s":0.972454,"scheduling_latency_s":0.972474,"redis_calls":14,"redis_duration_s":0.002074,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000326,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027093,"completed_at":"2026-07-04T10:54:06.904Z","load_balancing_strategy":"primary","db_duration_s":0.001935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586vm-test-run-nixbot-gitlab> gitlab # [  260.688750] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:06.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fbad9bb38fa040190705e03d","created_at":"2026-07-04T10:54:05.924Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"105aef45f8b1aae76e460f3e5f10305a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.933Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fbad9bb38fa040190705e03d: start","job_status":"start","queue_duration_s":0.971879,"scheduling_latency_s":0.971896}1587vm-test-run-nixbot-gitlab> gitlab # [  260.703035] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fbad9bb38fa040190705e03d","created_at":"2026-07-04T10:54:05.924Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"105aef45f8b1aae76e460f3e5f10305a","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.933Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fbad9bb38fa040190705e03d: done: 0.314918 sec","job_status":"done","queue_duration_s":0.971879,"scheduling_latency_s":0.971896,"redis_calls":571,"redis_duration_s":0.058976,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.057274,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.293258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.314918,"completed_at":"2026-07-04T10:54:07.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588vm-test-run-nixbot-gitlab> gitlab # [  260.744531] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"73cfea271f86524075552e3e","created_at":"2026-07-04T10:54:05.954Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab83e3798777007a1b001ba93c39f68e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.964Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-73cfea271f86524075552e3e: start","job_status":"start","queue_duration_s":1.257455,"scheduling_latency_s":1.25748}1589vm-test-run-nixbot-gitlab> gitlab # [  260.759151] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.230Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1590vm-test-run-nixbot-gitlab> gitlab # [  260.762844] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"73cfea271f86524075552e3e","created_at":"2026-07-04T10:54:05.954Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-56ab485c93e84f25","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab83e3798777007a1b001ba93c39f68e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:05.964Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-73cfea271f86524075552e3e: done: 0.014235 sec","job_status":"done","queue_duration_s":1.257455,"scheduling_latency_s":1.25748,"redis_calls":7,"redis_duration_s":0.001129,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014235,"completed_at":"2026-07-04T10:54:07.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1591vm-test-run-nixbot-gitlab> gitlab # [  260.805105] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66a18eba3491005debced059","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66a18eba3491005debced059: start","job_status":"start","queue_duration_s":1.198816,"scheduling_latency_s":1.198833}1592vm-test-run-nixbot-gitlab> gitlab # [  260.817641] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66a18eba3491005debced059","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66a18eba3491005debced059: done: 0.015917 sec","job_status":"done","queue_duration_s":1.198816,"scheduling_latency_s":1.198833,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015917,"completed_at":"2026-07-04T10:54:07.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1593vm-test-run-nixbot-gitlab> gitlab # [  260.857244] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13012a2e6649b9a41852b2cb","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13012a2e6649b9a41852b2cb: start","job_status":"start","queue_duration_s":1.216882,"scheduling_latency_s":1.216901}1594vm-test-run-nixbot-gitlab> gitlab # [  260.870052] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13012a2e6649b9a41852b2cb","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13012a2e6649b9a41852b2cb: done: 0.015573 sec","job_status":"done","queue_duration_s":1.216882,"scheduling_latency_s":1.216901,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015573,"completed_at":"2026-07-04T10:54:07.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1595vm-test-run-nixbot-gitlab> gitlab # [  260.909684] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd8242b3861ecab1f6f4fcd5","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd8242b3861ecab1f6f4fcd5: start","job_status":"start","queue_duration_s":1.233588,"scheduling_latency_s":1.233607}1596vm-test-run-nixbot-gitlab> gitlab # [  260.922324] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd8242b3861ecab1f6f4fcd5","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd8242b3861ecab1f6f4fcd5: done: 0.015774 sec","job_status":"done","queue_duration_s":1.233588,"scheduling_latency_s":1.233607,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015774,"completed_at":"2026-07-04T10:54:07.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1597vm-test-run-nixbot-gitlab> gitlab # [  260.961990] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440c6df99dd4754021609382","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440c6df99dd4754021609382: start","job_status":"start","queue_duration_s":1.250496,"scheduling_latency_s":1.250514}1598vm-test-run-nixbot-gitlab> gitlab # [  260.974517] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440c6df99dd4754021609382","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440c6df99dd4754021609382: done: 0.015056 sec","job_status":"done","queue_duration_s":1.250496,"scheduling_latency_s":1.250514,"redis_calls":13,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015056,"completed_at":"2026-07-04T10:54:07.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599vm-test-run-nixbot-gitlab> gitlab # [  261.015646] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fa40d7fd20e79b332a9cd8","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fa40d7fd20e79b332a9cd8: start","job_status":"start","queue_duration_s":1.266678,"scheduling_latency_s":1.266695}1600vm-test-run-nixbot-gitlab> gitlab # [  261.028461] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fa40d7fd20e79b332a9cd8","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fa40d7fd20e79b332a9cd8: done: 0.016493 sec","job_status":"done","queue_duration_s":1.266678,"scheduling_latency_s":1.266695,"redis_calls":13,"redis_duration_s":0.001697,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016493,"completed_at":"2026-07-04T10:54:07.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1601vm-test-run-nixbot-gitlab> gitlab # [  261.068295] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13b0557913ff011b6281911f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13b0557913ff011b6281911f: start","job_status":"start","queue_duration_s":1.285149,"scheduling_latency_s":1.285168}1602vm-test-run-nixbot-gitlab> gitlab # [  261.080941] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13b0557913ff011b6281911f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13b0557913ff011b6281911f: done: 0.015839 sec","job_status":"done","queue_duration_s":1.285149,"scheduling_latency_s":1.285168,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015839,"completed_at":"2026-07-04T10:54:07.339Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1603vm-test-run-nixbot-gitlab> gitlab # [  261.120880] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47e0a8f985864d6d1013646","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47e0a8f985864d6d1013646: start","job_status":"start","queue_duration_s":1.30221,"scheduling_latency_s":1.302227}1604vm-test-run-nixbot-gitlab> gitlab # [  261.133497] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47e0a8f985864d6d1013646","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47e0a8f985864d6d1013646: done: 0.015786 sec","job_status":"done","queue_duration_s":1.30221,"scheduling_latency_s":1.302227,"redis_calls":13,"redis_duration_s":0.001702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015786,"completed_at":"2026-07-04T10:54:07.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605vm-test-run-nixbot-gitlab> gitlab # [  261.173246] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cd36df45665c4d490ac25bf","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cd36df45665c4d490ac25bf: start","job_status":"start","queue_duration_s":1.319073,"scheduling_latency_s":1.31909}1606vm-test-run-nixbot-gitlab> gitlab # [  261.185659] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cd36df45665c4d490ac25bf","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cd36df45665c4d490ac25bf: done: 0.015146 sec","job_status":"done","queue_duration_s":1.319073,"scheduling_latency_s":1.31909,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015146,"completed_at":"2026-07-04T10:54:07.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1607vm-test-run-nixbot-gitlab> gitlab # [  261.226407] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cba8ced84c439bcd10b9d00","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cba8ced84c439bcd10b9d00: start","job_status":"start","queue_duration_s":1.335566,"scheduling_latency_s":1.335584}1608vm-test-run-nixbot-gitlab> gitlab # [  261.239615] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7cba8ced84c439bcd10b9d00","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7cba8ced84c439bcd10b9d00: done: 0.015408 sec","job_status":"done","queue_duration_s":1.335566,"scheduling_latency_s":1.335584,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015408,"completed_at":"2026-07-04T10:54:07.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1609vm-test-run-nixbot-gitlab> gitlab # [  261.279756] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26c1f1a16a47561a7a8d1a78","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26c1f1a16a47561a7a8d1a78: start","job_status":"start","queue_duration_s":1.352874,"scheduling_latency_s":1.352892}1610vm-test-run-nixbot-gitlab> gitlab # [  261.292686] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26c1f1a16a47561a7a8d1a78","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26c1f1a16a47561a7a8d1a78: done: 0.015079 sec","job_status":"done","queue_duration_s":1.352874,"scheduling_latency_s":1.352892,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015079,"completed_at":"2026-07-04T10:54:07.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1611vm-test-run-nixbot-gitlab> gitlab # [  261.332184] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5b3952344d1a00a2e87813f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5b3952344d1a00a2e87813f: start","job_status":"start","queue_duration_s":1.369015,"scheduling_latency_s":1.369033}1612vm-test-run-nixbot-gitlab> gitlab # [  261.344789] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5b3952344d1a00a2e87813f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5b3952344d1a00a2e87813f: done: 0.015417 sec","job_status":"done","queue_duration_s":1.369015,"scheduling_latency_s":1.369033,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015417,"completed_at":"2026-07-04T10:54:07.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1613vm-test-run-nixbot-gitlab> gitlab # [  261.384845] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a9b03da7b4c25a64ff17146","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a9b03da7b4c25a64ff17146: start","job_status":"start","queue_duration_s":1.385483,"scheduling_latency_s":1.385501}1614vm-test-run-nixbot-gitlab> gitlab # [  261.398358] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a9b03da7b4c25a64ff17146","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a9b03da7b4c25a64ff17146: done: 0.01488 sec","job_status":"done","queue_duration_s":1.385483,"scheduling_latency_s":1.385501,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01488,"completed_at":"2026-07-04T10:54:07.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1615vm-test-run-nixbot-gitlab> gitlab # [  261.437999] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"285b7a60c84b80262b901510","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-285b7a60c84b80262b901510: start","job_status":"start","queue_duration_s":1.401392,"scheduling_latency_s":1.401409}1616vm-test-run-nixbot-gitlab> gitlab # [  261.450761] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"285b7a60c84b80262b901510","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-285b7a60c84b80262b901510: done: 0.014933 sec","job_status":"done","queue_duration_s":1.401392,"scheduling_latency_s":1.401409,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014933,"completed_at":"2026-07-04T10:54:07.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1617vm-test-run-nixbot-gitlab> gitlab # [  261.490376] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8a65cb156f263b555f23619","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8a65cb156f263b555f23619: start","job_status":"start","queue_duration_s":1.41969,"scheduling_latency_s":1.419709}1618vm-test-run-nixbot-gitlab> gitlab # [  261.503025] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8a65cb156f263b555f23619","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8a65cb156f263b555f23619: done: 0.019849 sec","job_status":"done","queue_duration_s":1.41969,"scheduling_latency_s":1.419709,"redis_calls":13,"redis_duration_s":0.001479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019849,"completed_at":"2026-07-04T10:54:07.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1619vm-test-run-nixbot-gitlab> gitlab # [  261.542814] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a503864c79883403859c9be","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a503864c79883403859c9be: start","job_status":"start","queue_duration_s":1.440596,"scheduling_latency_s":1.440614}1620vm-test-run-nixbot-gitlab> gitlab # [  261.555467] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a503864c79883403859c9be","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a503864c79883403859c9be: done: 0.015952 sec","job_status":"done","queue_duration_s":1.440596,"scheduling_latency_s":1.440614,"redis_calls":13,"redis_duration_s":0.0018380000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015952,"completed_at":"2026-07-04T10:54:07.494Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1621vm-test-run-nixbot-gitlab> gitlab # [  261.595539] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4afd34b31021d45e21849927","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4afd34b31021d45e21849927: start","job_status":"start","queue_duration_s":1.457765,"scheduling_latency_s":1.457784}1622vm-test-run-nixbot-gitlab> gitlab # [  261.609455] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4afd34b31021d45e21849927","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4afd34b31021d45e21849927: done: 0.01512 sec","job_status":"done","queue_duration_s":1.457765,"scheduling_latency_s":1.457784,"redis_calls":13,"redis_duration_s":0.0014329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01512,"completed_at":"2026-07-04T10:54:07.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1623vm-test-run-nixbot-gitlab> gitlab # [  261.649143] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"108f22d62260beafac302c71","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-108f22d62260beafac302c71: start","job_status":"start","queue_duration_s":1.473933,"scheduling_latency_s":1.47395}1624vm-test-run-nixbot-gitlab> gitlab # [  261.661636] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"108f22d62260beafac302c71","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-108f22d62260beafac302c71: done: 0.015835 sec","job_status":"done","queue_duration_s":1.473933,"scheduling_latency_s":1.47395,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015835,"completed_at":"2026-07-04T10:54:07.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1625vm-test-run-nixbot-gitlab> gitlab # [  261.701356] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5aa6502add009293536ae7e3","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5aa6502add009293536ae7e3: start","job_status":"start","queue_duration_s":1.490924,"scheduling_latency_s":1.490942}1626vm-test-run-nixbot-gitlab> gitlab # [  261.713902] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5aa6502add009293536ae7e3","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5aa6502add009293536ae7e3: done: 0.015252 sec","job_status":"done","queue_duration_s":1.490924,"scheduling_latency_s":1.490942,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015252,"completed_at":"2026-07-04T10:54:07.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1627vm-test-run-nixbot-gitlab> gitlab # [  261.753623] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62a54d3afae9d2cda6523c21","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62a54d3afae9d2cda6523c21: start","job_status":"start","queue_duration_s":1.507257,"scheduling_latency_s":1.507275}1628vm-test-run-nixbot-gitlab> gitlab # [  261.766356] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62a54d3afae9d2cda6523c21","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62a54d3afae9d2cda6523c21: done: 0.014939 sec","job_status":"done","queue_duration_s":1.507257,"scheduling_latency_s":1.507275,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014939,"completed_at":"2026-07-04T10:54:07.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1629vm-test-run-nixbot-gitlab> gitlab # [  261.806842] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07be3c731a10b375f5bf589b","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07be3c731a10b375f5bf589b: start","job_status":"start","queue_duration_s":1.523271,"scheduling_latency_s":1.523288}1630vm-test-run-nixbot-gitlab> gitlab # [  261.819319] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07be3c731a10b375f5bf589b","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07be3c731a10b375f5bf589b: done: 0.015053 sec","job_status":"done","queue_duration_s":1.523271,"scheduling_latency_s":1.523288,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015053,"completed_at":"2026-07-04T10:54:07.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1631vm-test-run-nixbot-gitlab> gitlab # [  261.859892] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62e4f0d2cc350eba5b3338b","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62e4f0d2cc350eba5b3338b: start","job_status":"start","queue_duration_s":1.540321,"scheduling_latency_s":1.540338}1632vm-test-run-nixbot-gitlab> gitlab # [  261.898140] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b62e4f0d2cc350eba5b3338b","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b62e4f0d2cc350eba5b3338b: done: 0.015364 sec","job_status":"done","queue_duration_s":1.540321,"scheduling_latency_s":1.540338,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015364,"completed_at":"2026-07-04T10:54:07.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1633vm-test-run-nixbot-gitlab> gitlab # [  261.988191] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8669fe389ba51d4df26e89c9","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8669fe389ba51d4df26e89c9: start","job_status":"start","queue_duration_s":1.556716,"scheduling_latency_s":1.556734}1634vm-test-run-nixbot-gitlab> gitlab # [  262.009599] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8669fe389ba51d4df26e89c9","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8669fe389ba51d4df26e89c9: done: 0.015047 sec","job_status":"done","queue_duration_s":1.556716,"scheduling_latency_s":1.556734,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015047,"completed_at":"2026-07-04T10:54:07.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1635vm-test-run-nixbot-gitlab> gitlab # [  262.069631] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08ab7c2033e4b698f5d5b005","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08ab7c2033e4b698f5d5b005: start","job_status":"start","queue_duration_s":1.572845,"scheduling_latency_s":1.572863}1636vm-test-run-nixbot-gitlab> gitlab # [  262.083749] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08ab7c2033e4b698f5d5b005","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08ab7c2033e4b698f5d5b005: done: 0.016075 sec","job_status":"done","queue_duration_s":1.572845,"scheduling_latency_s":1.572863,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016075,"completed_at":"2026-07-04T10:54:07.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1637vm-test-run-nixbot-gitlab> gitlab # [  262.124743] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2425f8614833c253dd6744","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2425f8614833c253dd6744: start","job_status":"start","queue_duration_s":1.590019,"scheduling_latency_s":1.590037}1638vm-test-run-nixbot-gitlab> gitlab # [  262.137269] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d2425f8614833c253dd6744","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d2425f8614833c253dd6744: done: 0.014946 sec","job_status":"done","queue_duration_s":1.590019,"scheduling_latency_s":1.590037,"redis_calls":13,"redis_duration_s":0.0014490000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014946,"completed_at":"2026-07-04T10:54:07.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1639vm-test-run-nixbot-gitlab> gitlab # [  262.177309] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2edc4ab266a59a5d76d5e159","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2edc4ab266a59a5d76d5e159: start","job_status":"start","queue_duration_s":1.606033,"scheduling_latency_s":1.60605}1640vm-test-run-nixbot-gitlab> gitlab # [  262.189833] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2edc4ab266a59a5d76d5e159","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2edc4ab266a59a5d76d5e159: done: 0.015166 sec","job_status":"done","queue_duration_s":1.606033,"scheduling_latency_s":1.60605,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015166,"completed_at":"2026-07-04T10:54:07.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1641vm-test-run-nixbot-gitlab> gitlab # [  262.230772] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e77226e7fce92b69ba351b60","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e77226e7fce92b69ba351b60: start","job_status":"start","queue_duration_s":1.623242,"scheduling_latency_s":1.62326}1642vm-test-run-nixbot-gitlab> gitlab # [  262.244104] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e77226e7fce92b69ba351b60","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e77226e7fce92b69ba351b60: done: 0.015086 sec","job_status":"done","queue_duration_s":1.623242,"scheduling_latency_s":1.62326,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015086,"completed_at":"2026-07-04T10:54:07.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1643vm-test-run-nixbot-gitlab> gitlab # [  262.284124] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8870b823d311a1296d50e0","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8870b823d311a1296d50e0: start","job_status":"start","queue_duration_s":1.639363,"scheduling_latency_s":1.63938}1644vm-test-run-nixbot-gitlab> gitlab # [  262.296850] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab8870b823d311a1296d50e0","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab8870b823d311a1296d50e0: done: 0.015378 sec","job_status":"done","queue_duration_s":1.639363,"scheduling_latency_s":1.63938,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015378,"completed_at":"2026-07-04T10:54:07.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1645vm-test-run-nixbot-gitlab> gitlab # [  262.336971] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c11b104b62e72fabcdd4f5b7","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c11b104b62e72fabcdd4f5b7: start","job_status":"start","queue_duration_s":1.65582,"scheduling_latency_s":1.655837}1646vm-test-run-nixbot-gitlab> gitlab # [  262.349536] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c11b104b62e72fabcdd4f5b7","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c11b104b62e72fabcdd4f5b7: done: 0.016131 sec","job_status":"done","queue_duration_s":1.65582,"scheduling_latency_s":1.655837,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016131,"completed_at":"2026-07-04T10:54:07.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1647vm-test-run-nixbot-gitlab> gitlab # [  262.389534] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5287e7151697d7c97d3e85cf","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5287e7151697d7c97d3e85cf: start","job_status":"start","queue_duration_s":1.672995,"scheduling_latency_s":1.673012}1648vm-test-run-nixbot-gitlab> gitlab # [  262.402374] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5287e7151697d7c97d3e85cf","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5287e7151697d7c97d3e85cf: done: 0.015074 sec","job_status":"done","queue_duration_s":1.672995,"scheduling_latency_s":1.673012,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015074,"completed_at":"2026-07-04T10:54:07.726Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1649vm-test-run-nixbot-gitlab> gitlab # [  262.443147] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9834cae28aa9326f2a324ed","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9834cae28aa9326f2a324ed: start","job_status":"start","queue_duration_s":1.68912,"scheduling_latency_s":1.689137}1650vm-test-run-nixbot-gitlab> gitlab # [  262.456000] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9834cae28aa9326f2a324ed","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9834cae28aa9326f2a324ed: done: 0.015003 sec","job_status":"done","queue_duration_s":1.68912,"scheduling_latency_s":1.689137,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015003,"completed_at":"2026-07-04T10:54:07.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1651vm-test-run-nixbot-gitlab> gitlab # [  262.496141] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48928b1db1191fefa9e9358f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48928b1db1191fefa9e9358f: start","job_status":"start","queue_duration_s":1.705172,"scheduling_latency_s":1.705189}1652vm-test-run-nixbot-gitlab> gitlab # [  262.508569] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48928b1db1191fefa9e9358f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48928b1db1191fefa9e9358f: done: 0.014754 sec","job_status":"done","queue_duration_s":1.705172,"scheduling_latency_s":1.705189,"redis_calls":13,"redis_duration_s":0.001409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014754,"completed_at":"2026-07-04T10:54:07.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1653vm-test-run-nixbot-gitlab> gitlab # [  262.548544] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00852c4f27ce36e4c0cf827d","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00852c4f27ce36e4c0cf827d: start","job_status":"start","queue_duration_s":1.720983,"scheduling_latency_s":1.720999}1654vm-test-run-nixbot-gitlab> gitlab # [  262.561065] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00852c4f27ce36e4c0cf827d","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00852c4f27ce36e4c0cf827d: done: 0.014538 sec","job_status":"done","queue_duration_s":1.720983,"scheduling_latency_s":1.720999,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014538,"completed_at":"2026-07-04T10:54:07.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1655vm-test-run-nixbot-gitlab> gitlab # [  262.601000] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c4eb554c9e391c2c3189f44","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c4eb554c9e391c2c3189f44: start","job_status":"start","queue_duration_s":1.73659,"scheduling_latency_s":1.736606}1656vm-test-run-nixbot-gitlab> gitlab # [  262.613752] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c4eb554c9e391c2c3189f44","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c4eb554c9e391c2c3189f44: done: 0.014887 sec","job_status":"done","queue_duration_s":1.73659,"scheduling_latency_s":1.736606,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014887,"completed_at":"2026-07-04T10:54:07.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1657vm-test-run-nixbot-gitlab> gitlab # [  262.655526] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb159f6aa37d89adfdd62d9f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb159f6aa37d89adfdd62d9f: start","job_status":"start","queue_duration_s":1.752956,"scheduling_latency_s":1.752973}1658vm-test-run-nixbot-gitlab> gitlab # [  262.668793] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb159f6aa37d89adfdd62d9f","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb159f6aa37d89adfdd62d9f: done: 0.01445 sec","job_status":"done","queue_duration_s":1.752956,"scheduling_latency_s":1.752973,"redis_calls":13,"redis_duration_s":0.0018080000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01445,"completed_at":"2026-07-04T10:54:07.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1659vm-test-run-nixbot-gitlab> gitlab # [  262.709018] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa548ba3085f6d2ea82f223c","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa548ba3085f6d2ea82f223c: start","job_status":"start","queue_duration_s":1.768485,"scheduling_latency_s":1.768502}1660vm-test-run-nixbot-gitlab> gitlab # [  262.721673] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa548ba3085f6d2ea82f223c","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa548ba3085f6d2ea82f223c: done: 0.014744 sec","job_status":"done","queue_duration_s":1.768485,"scheduling_latency_s":1.768502,"redis_calls":13,"redis_duration_s":0.0015730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014744,"completed_at":"2026-07-04T10:54:07.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1661vm-test-run-nixbot-gitlab> gitlab # [  262.761843] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ebf9546de10bfca7aedcb70","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ebf9546de10bfca7aedcb70: start","job_status":"start","queue_duration_s":1.78425,"scheduling_latency_s":1.784268}1662vm-test-run-nixbot-gitlab> gitlab # [  262.774783] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ebf9546de10bfca7aedcb70","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ebf9546de10bfca7aedcb70: done: 0.01454 sec","job_status":"done","queue_duration_s":1.78425,"scheduling_latency_s":1.784268,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01454,"completed_at":"2026-07-04T10:54:07.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1663vm-test-run-nixbot-gitlab> gitlab # [  262.814630] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feef07deabe089524d34f72c","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feef07deabe089524d34f72c: start","job_status":"start","queue_duration_s":1.800249,"scheduling_latency_s":1.800267}1664vm-test-run-nixbot-gitlab> gitlab # [  262.828803] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feef07deabe089524d34f72c","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feef07deabe089524d34f72c: done: 0.01449 sec","job_status":"done","queue_duration_s":1.800249,"scheduling_latency_s":1.800267,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01449,"completed_at":"2026-07-04T10:54:07.852Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1665vm-test-run-nixbot-gitlab> gitlab # [  262.922399] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb658cb65b943875f720fff5","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb658cb65b943875f720fff5: start","job_status":"start","queue_duration_s":1.815775,"scheduling_latency_s":1.815793}1666vm-test-run-nixbot-gitlab> gitlab # [  262.958036] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb658cb65b943875f720fff5","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb658cb65b943875f720fff5: done: 0.014669 sec","job_status":"done","queue_duration_s":1.815775,"scheduling_latency_s":1.815793,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014669,"completed_at":"2026-07-04T10:54:07.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1667vm-test-run-nixbot-gitlab> gitlab # [  263.019082] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fde05ad6b1abc01ba5f6c0a9","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fde05ad6b1abc01ba5f6c0a9: start","job_status":"start","queue_duration_s":1.831521,"scheduling_latency_s":1.831539}1668vm-test-run-nixbot-gitlab> gitlab # [  263.033734] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fde05ad6b1abc01ba5f6c0a9","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fde05ad6b1abc01ba5f6c0a9: done: 0.014484 sec","job_status":"done","queue_duration_s":1.831521,"scheduling_latency_s":1.831539,"redis_calls":13,"redis_duration_s":0.0014299999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014484,"completed_at":"2026-07-04T10:54:07.884Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1669vm-test-run-nixbot-gitlab> gitlab # [  263.082826] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a73dcef27aff95ae76243a58","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a73dcef27aff95ae76243a58: start","job_status":"start","queue_duration_s":1.847061,"scheduling_latency_s":1.847078}1670vm-test-run-nixbot-gitlab> gitlab # [  263.096649] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a73dcef27aff95ae76243a58","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a73dcef27aff95ae76243a58: done: 0.015681 sec","job_status":"done","queue_duration_s":1.847061,"scheduling_latency_s":1.847078,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015681,"completed_at":"2026-07-04T10:54:07.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1671vm-test-run-nixbot-gitlab> gitlab # [  263.136735] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d77a4ebca6ea02538740f58","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d77a4ebca6ea02538740f58: start","job_status":"start","queue_duration_s":1.86382,"scheduling_latency_s":1.863838}1672vm-test-run-nixbot-gitlab> gitlab # [  263.149282] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d77a4ebca6ea02538740f58","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d77a4ebca6ea02538740f58: done: 0.014664 sec","job_status":"done","queue_duration_s":1.86382,"scheduling_latency_s":1.863838,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014664,"completed_at":"2026-07-04T10:54:07.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1673vm-test-run-nixbot-gitlab> gitlab # [  263.189178] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8648374ce38b7bd81f61d099","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8648374ce38b7bd81f61d099: start","job_status":"start","queue_duration_s":1.879621,"scheduling_latency_s":1.879639}1674vm-test-run-nixbot-gitlab> gitlab # [  263.201955] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8648374ce38b7bd81f61d099","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8648374ce38b7bd81f61d099: done: 0.014294 sec","job_status":"done","queue_duration_s":1.879621,"scheduling_latency_s":1.879639,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014294,"completed_at":"2026-07-04T10:54:07.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1675vm-test-run-nixbot-gitlab> gitlab # [  263.242522] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a8fdd332edf2955417722ad","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a8fdd332edf2955417722ad: start","job_status":"start","queue_duration_s":1.894981,"scheduling_latency_s":1.894999}1676vm-test-run-nixbot-gitlab> gitlab # [  263.255743] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3a8fdd332edf2955417722ad","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3a8fdd332edf2955417722ad: done: 0.014321 sec","job_status":"done","queue_duration_s":1.894981,"scheduling_latency_s":1.894999,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014321,"completed_at":"2026-07-04T10:54:07.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1677vm-test-run-nixbot-gitlab> gitlab # [  263.295844] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd651000663f67a60789ec33","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd651000663f67a60789ec33: start","job_status":"start","queue_duration_s":1.910713,"scheduling_latency_s":1.91073}1678vm-test-run-nixbot-gitlab> gitlab # [  263.308595] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd651000663f67a60789ec33","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd651000663f67a60789ec33: done: 0.014706 sec","job_status":"done","queue_duration_s":1.910713,"scheduling_latency_s":1.91073,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014706,"completed_at":"2026-07-04T10:54:07.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1679vm-test-run-nixbot-gitlab> gitlab # [  263.348708] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"255d2b15da12e51f61546d03","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-255d2b15da12e51f61546d03: start","job_status":"start","queue_duration_s":1.926524,"scheduling_latency_s":1.926541}1680vm-test-run-nixbot-gitlab> gitlab # [  263.361146] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"255d2b15da12e51f61546d03","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-255d2b15da12e51f61546d03: done: 0.013979 sec","job_status":"done","queue_duration_s":1.926524,"scheduling_latency_s":1.926541,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013979,"completed_at":"2026-07-04T10:54:07.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1681vm-test-run-nixbot-gitlab> gitlab # [  263.400713] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84994e5fb276b1c8c648c1c2","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84994e5fb276b1c8c648c1c2: start","job_status":"start","queue_duration_s":1.941572,"scheduling_latency_s":1.941591}1682vm-test-run-nixbot-gitlab> gitlab # [  263.413253] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84994e5fb276b1c8c648c1c2","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84994e5fb276b1c8c648c1c2: done: 0.01503 sec","job_status":"done","queue_duration_s":1.941572,"scheduling_latency_s":1.941591,"redis_calls":13,"redis_duration_s":0.0015960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01503,"completed_at":"2026-07-04T10:54:07.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1683vm-test-run-nixbot-gitlab> gitlab # [  263.454038] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:07.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4590291a1b89523affddbd","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4590291a1b89523affddbd: start","job_status":"start","queue_duration_s":1.95765,"scheduling_latency_s":1.957667}1684vm-test-run-nixbot-gitlab> gitlab # [  263.466622] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4590291a1b89523affddbd","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4590291a1b89523affddbd: done: 0.014289 sec","job_status":"done","queue_duration_s":1.95765,"scheduling_latency_s":1.957667,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014289,"completed_at":"2026-07-04T10:54:08.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1685vm-test-run-nixbot-gitlab> gitlab # [  263.506046] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ebbc78d19b45c2bedd2d3d1","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ebbc78d19b45c2bedd2d3d1: start","job_status":"start","queue_duration_s":1.972977,"scheduling_latency_s":1.972994}1686vm-test-run-nixbot-gitlab> gitlab # [  263.518781] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ebbc78d19b45c2bedd2d3d1","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ebbc78d19b45c2bedd2d3d1: done: 0.014633 sec","job_status":"done","queue_duration_s":1.972977,"scheduling_latency_s":1.972994,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014633,"completed_at":"2026-07-04T10:54:08.025Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1687vm-test-run-nixbot-gitlab> gitlab # [  263.558249] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.026Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85b9a23c94cb34e410bb3406","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85b9a23c94cb34e410bb3406: start","job_status":"start","queue_duration_s":1.988705,"scheduling_latency_s":1.988723}1688vm-test-run-nixbot-gitlab> gitlab # [  263.570781] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85b9a23c94cb34e410bb3406","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85b9a23c94cb34e410bb3406: done: 0.014445 sec","job_status":"done","queue_duration_s":1.988705,"scheduling_latency_s":1.988723,"redis_calls":13,"redis_duration_s":0.001452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014445,"completed_at":"2026-07-04T10:54:08.041Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1689vm-test-run-nixbot-gitlab> gitlab # [  263.610371] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa7742aa5a0979f8ed3100c6","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa7742aa5a0979f8ed3100c6: start","job_status":"start","queue_duration_s":2.00422,"scheduling_latency_s":2.004237}1690vm-test-run-nixbot-gitlab> gitlab # [  263.622852] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa7742aa5a0979f8ed3100c6","created_at":"2026-07-04T10:54:05.496Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-4a9a61d10d3147e4","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cf56b37655464e7e342b1afa71da8e0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.038Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa7742aa5a0979f8ed3100c6: done: 0.014304 sec","job_status":"done","queue_duration_s":2.00422,"scheduling_latency_s":2.004237,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014304,"completed_at":"2026-07-04T10:54:08.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1691vm-test-run-nixbot-gitlab> gitlab # [  263.663533] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.057Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf2b53de7dc680fc4c478511","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf2b53de7dc680fc4c478511: start","job_status":"start","queue_duration_s":1.160051,"scheduling_latency_s":1.160068}1692vm-test-run-nixbot-gitlab> gitlab # [  263.676226] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.069Z","message":"No existing merge request to be cleaned up."}1693vm-test-run-nixbot-gitlab> gitlab # [  263.677924] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.078Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cf2b53de7dc680fc4c478511","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cf2b53de7dc680fc4c478511: done: 0.020499 sec","job_status":"done","queue_duration_s":1.160051,"scheduling_latency_s":1.160068,"redis_calls":10,"redis_duration_s":0.001415,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020499,"completed_at":"2026-07-04T10:54:08.078Z","load_balancing_strategy":"primary","db_duration_s":0.003735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1694vm-test-run-nixbot-gitlab> gitlab # [  263.716964] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.079Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"31a1c364da9519e70ca7c1aa","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-31a1c364da9519e70ca7c1aa: start","job_status":"start","queue_duration_s":1.181625,"scheduling_latency_s":1.181642}1695vm-test-run-nixbot-gitlab> gitlab # [  263.729419] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.089Z","message":"No existing merge request to be cleaned up."}1696vm-test-run-nixbot-gitlab> gitlab # [  263.731085] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.097Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"31a1c364da9519e70ca7c1aa","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-31a1c364da9519e70ca7c1aa: done: 0.018202 sec","job_status":"done","queue_duration_s":1.181625,"scheduling_latency_s":1.181642,"redis_calls":10,"redis_duration_s":0.0013219999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000761,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018202,"completed_at":"2026-07-04T10:54:08.097Z","load_balancing_strategy":"primary","db_duration_s":0.002342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1697vm-test-run-nixbot-gitlab> gitlab # [  263.770143] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.098Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f8333d577e59229e2f7f71c7","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f8333d577e59229e2f7f71c7: start","job_status":"start","queue_duration_s":1.201102,"scheduling_latency_s":1.201119}1698vm-test-run-nixbot-gitlab> gitlab # [  263.782756] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.109Z","message":"No existing merge request to be cleaned up."}1699vm-test-run-nixbot-gitlab> gitlab # [  263.784513] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.117Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f8333d577e59229e2f7f71c7","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f8333d577e59229e2f7f71c7: done: 0.018261 sec","job_status":"done","queue_duration_s":1.201102,"scheduling_latency_s":1.201119,"redis_calls":10,"redis_duration_s":0.0013579999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018261,"completed_at":"2026-07-04T10:54:08.117Z","load_balancing_strategy":"primary","db_duration_s":0.002283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1700vm-test-run-nixbot-gitlab> gitlab # [  263.823777] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.118Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07292b3c609228eff5f3358c","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07292b3c609228eff5f3358c: start","job_status":"start","queue_duration_s":1.220508,"scheduling_latency_s":1.220526}1701vm-test-run-nixbot-gitlab> gitlab # [  263.861671] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.128Z","message":"No existing merge request to be cleaned up."}1702vm-test-run-nixbot-gitlab> gitlab # [  263.867800] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:08.136Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07292b3c609228eff5f3358c","created_at":"2026-07-04T10:54:06.888Z","trace_propagation_headers":{"sentry-trace":"f4880b24cc56443e8a103813882d0c19-d17d077302664d01","baggage":"sentry-trace_id=f4880b24cc56443e8a103813882d0c19,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"cdb452c58bbb0e5c1c0062369e0a249c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:06.897Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07292b3c609228eff5f3358c: done: 0.018109 sec","job_status":"done","queue_duration_s":1.220508,"scheduling_latency_s":1.220526,"redis_calls":10,"redis_duration_s":0.001337,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018109,"completed_at":"2026-07-04T10:54:08.136Z","load_balancing_strategy":"primary","db_duration_s":0.00217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1703vm-test-run-nixbot-gitlab> gitlab # [  293.750843] runuser[3291]: pam_unix(runuser:session): session closed for user gitlab1704vm-test-run-nixbot-gitlab> gitlab # [  293.753874] setup-gitlab-start[3622]: ++ jq -r .token1705vm-test-run-nixbot-gitlab> gitlab # [  293.755001] setup-gitlab-start[3623]: +++ date -d '+7 days' +%F1706vm-test-run-nixbot-gitlab> gitlab # [  293.760577] setup-gitlab-start[3621]: ++ curl -fs -X POST -H 'Authorization: Bearer ffd715455da63ed7f3a75aedcd2b2fe85d92c167afb2196e09179b85c0f8343d' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-11"}' http://gitlab/api/v4/users/1/personal_access_tokens1707vm-test-run-nixbot-gitlab> gitlab # [  294.455089] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:41.285Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"10e2d50a71659ec712d4680b","created_at":"2026-07-04T10:54:41.253Z","trace_propagation_headers":{"sentry-trace":"f798e46bbf9c4032b3943f0f77db1564-6454fd30caae4d76","baggage":"sentry-trace_id=f798e46bbf9c4032b3943f0f77db1564,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC89Z8Y0J18MMDTDK5CF6V","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2011343ad259f9b0f36611a4ad432a35e375d93940e8539715b411c1fade5d04","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:41.281Z","job_size_bytes":451,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-10e2d50a71659ec712d4680b: start","job_status":"start","queue_duration_s":0.003422,"scheduling_latency_s":0.003459}1708vm-test-run-nixbot-gitlab> gitlab # [  294.536334] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:41 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 7491709vm-test-run-nixbot-gitlab> gitlab # [  294.543363] setup-gitlab-start[3240]: + TOKEN=glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck1710vm-test-run-nixbot-gitlab> gitlab # [  294.547146] setup-gitlab-start[3240]: + test -n glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck1711vm-test-run-nixbot-gitlab> gitlab # [  294.550899] setup-gitlab-start[3240]: + test glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck '!=' null1712vm-test-run-nixbot-gitlab> gitlab # [  294.554924] setup-gitlab-start[3240]: + mkdir -p /var/lib/secrets1713vm-test-run-nixbot-gitlab> gitlab # [  294.557622] setup-gitlab-start[3240]: + echo glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck1714vm-test-run-nixbot-gitlab> gitlab # [  294.561275] setup-gitlab-start[3240]: + chmod 644 /var/lib/secrets/gitlab-token1715vm-test-run-nixbot-gitlab> gitlab # [  294.564308] setup-gitlab-start[3240]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects1716vm-test-run-nixbot-gitlab> gitlab # [  294.618117] sidekiq-cluster[2431]: {"severity":"WARN","time":"2026-07-04T10:54:41.448Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"10e2d50a71659ec712d4680b","created_at":"2026-07-04T10:54:41.253Z","trace_propagation_headers":{"sentry-trace":"f798e46bbf9c4032b3943f0f77db1564-6454fd30caae4d76","baggage":"sentry-trace_id=f798e46bbf9c4032b3943f0f77db1564,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC89Z8Y0J18MMDTDK5CF6V","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2011343ad259f9b0f36611a4ad432a35e375d93940e8539715b411c1fade5d04","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:41.281Z","job_size_bytes":451,"pid":2431,"sidekiq_tid":"lylj","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-10e2d50a71659ec712d4680b: fail: 0.163112 sec","job_status":"fail","queue_duration_s":0.003422,"scheduling_latency_s":0.003459,"redis_calls":17,"redis_duration_s":0.004865,"redis_read_bytes":651,"redis_write_bytes":1842,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002703,"redis_feature_flag_read_bytes":644,"redis_feature_flag_write_bytes":958,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004939,"duration_s":0.163112,"completed_at":"2026-07-04T10:54:41.448Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.038748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1717vm-test-run-nixbot-gitlab> gitlab # [  295.366944] gitaly[1383]: time="2026-07-04T10:54:42.197Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.197" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718vm-test-run-nixbot-gitlab> gitlab # [  295.384820] gitaly[1383]: time="2026-07-04T10:54:42.198Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.197" grpc.time_ms=0.67 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1719vm-test-run-nixbot-gitlab> gitlab # [  295.621476] gitaly[1383]: time="2026-07-04T10:54:42.453Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.261" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.452" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720vm-test-run-nixbot-gitlab> gitlab # [  295.653581] gitaly[1383]: time="2026-07-04T10:54:42.453Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.261" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.452" grpc.time_ms=0.576 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721vm-test-run-nixbot-gitlab> gitlab # [  295.689206] gitaly[1383]: time="2026-07-04T10:54:42.498Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ diskcache=3448e725-f753-430b-bf68-48afb56ad7c9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T10:54:52.275" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.466" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1722vm-test-run-nixbot-gitlab> gitlab # [  295.717468] gitaly[1383]: time="2026-07-04T10:54:42.499Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=298 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T10:54:52.275" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.466" grpc.time_ms=32.372 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1723vm-test-run-nixbot-gitlab> gitlab # [  295.747393] gitaly[1383]: time="2026-07-04T10:54:42.520Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.519" pid=1383 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1724vm-test-run-nixbot-gitlab> gitlab # [  295.766791] gitaly[1383]: time="2026-07-04T10:54:42.521Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.519" grpc.time_ms=1.401 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1725vm-test-run-nixbot-gitlab> gitlab # [  295.785456] gitaly[1383]: time="2026-07-04T10:54:42.546Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:52.017" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.528" grpc.time_ms=17.349 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1726vm-test-run-nixbot-gitlab> gitlab # [  295.805094] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:42.584Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"35945e490e3716a53ac19a17","created_at":"2026-07-04T10:54:42.561Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:42.581Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-35945e490e3716a53ac19a17: start","job_status":"start","queue_duration_s":0.002939,"scheduling_latency_s":0.003004}1727vm-test-run-nixbot-gitlab> gitlab # [  295.828986] gitaly[1383]: time="2026-07-04T10:54:42.621Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.110" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.620" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1728vm-test-run-nixbot-gitlab> gitlab # [  295.842781] gitaly[1383]: time="2026-07-04T10:54:42.621Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.110" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.620" grpc.time_ms=0.471 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1729vm-test-run-nixbot-gitlab> gitlab # [  295.856135] gitaly[1383]: time="2026-07-04T10:54:42.648Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ diskcache=2d943b55-6e40-4586-858f-54e79683cd52 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T10:54:52.114" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.625" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1730vm-test-run-nixbot-gitlab> gitlab # [  295.867737] gitaly[1383]: time="2026-07-04T10:54:42.648Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=299 command.oublock=32 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-04T10:54:52.114" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.625" grpc.time_ms=22.878 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1731vm-test-run-nixbot-gitlab> gitlab # [  295.883193] gitaly[1383]: time="2026-07-04T10:54:42.660Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.149" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.659" pid=1383 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1732vm-test-run-nixbot-gitlab> gitlab # [  295.894889] gitaly[1383]: time="2026-07-04T10:54:42.661Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.149" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.659" grpc.time_ms=1.242 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1733vm-test-run-nixbot-gitlab> gitlab # [  295.907478] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:42.706Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"35945e490e3716a53ac19a17","created_at":"2026-07-04T10:54:42.561Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:42.581Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-35945e490e3716a53ac19a17: done: 0.122162 sec","job_status":"done","queue_duration_s":0.002939,"scheduling_latency_s":0.003004,"redis_calls":15,"redis_duration_s":0.0071730000000000006,"redis_read_bytes":140,"redis_write_bytes":1876,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.002006,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":711,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002128,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06642014,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.122162,"completed_at":"2026-07-04T10:54:42.706Z","load_balancing_strategy":"primary","db_duration_s":0.013823,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1734vm-test-run-nixbot-gitlab> gitlab # [  295.953466] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:42.708Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f1793bb2eea3e58b4ab1b50f","created_at":"2026-07-04T10:54:42.603Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:42.613Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f1793bb2eea3e58b4ab1b50f: start","job_status":"start","queue_duration_s":0.094869,"scheduling_latency_s":0.094895}1735vm-test-run-nixbot-gitlab> gitlab # [  295.968131] gitaly[1383]: time="2026-07-04T10:54:42.682Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:52.157" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.667" grpc.time_ms=14.5 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1736vm-test-run-nixbot-gitlab> gitlab # [  295.982188] gitaly[1383]: time="2026-07-04T10:54:42.732Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:52.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.731" grpc.time_ms=0.302 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1737vm-test-run-nixbot-gitlab> gitlab # [  295.993411] gitaly[1383]: time="2026-07-04T10:54:42.745Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T10:54:52.234" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:42.744" grpc.time_ms=0.5 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1738vm-test-run-nixbot-gitlab> gitlab # [  296.083217] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:42.914Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f1793bb2eea3e58b4ab1b50f","created_at":"2026-07-04T10:54:42.603Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:42.613Z","job_size_bytes":2,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f1793bb2eea3e58b4ab1b50f: done: 0.205784 sec","job_status":"done","queue_duration_s":0.094869,"scheduling_latency_s":0.094895,"redis_calls":14,"redis_duration_s":0.004328,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001254,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001719,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.15622282,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.205784,"completed_at":"2026-07-04T10:54:42.914Z","load_balancing_strategy":"primary","db_duration_s":0.048607,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1739vm-test-run-nixbot-gitlab> gitlab # [  296.394210] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.224Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0080bdf230ba3183431d2e2d","created_at":"2026-07-04T10:54:43.211Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.222Z","job_size_bytes":5,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0080bdf230ba3183431d2e2d: start","job_status":"start","queue_duration_s":0.002461,"scheduling_latency_s":0.002525}1740vm-test-run-nixbot-gitlab> gitlab # [  296.528745] gitaly[1383]: time="2026-07-04T10:54:43.359Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8AQM52P0ZYDQ50YJ17XZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:53.150" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:43.341" grpc.time_ms=17.496 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1741vm-test-run-nixbot-gitlab> gitlab # [  296.637706] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.468Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0080bdf230ba3183431d2e2d","created_at":"2026-07-04T10:54:43.211Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.222Z","job_size_bytes":5,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0080bdf230ba3183431d2e2d: done: 0.243425 sec","job_status":"done","queue_duration_s":0.002461,"scheduling_latency_s":0.002525,"redis_calls":12,"redis_duration_s":0.004888,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001001,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003154,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.195715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079374,"duration_s":0.243425,"completed_at":"2026-07-04T10:54:43.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03484,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1742vm-test-run-nixbot-gitlab> gitlab # [  296.731808] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.471Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ef1b99f71d9cf18898aef080","created_at":"2026-07-04T10:54:43.275Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.290Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ef1b99f71d9cf18898aef080: start","job_status":"start","queue_duration_s":0.181099,"scheduling_latency_s":0.181139}1743vm-test-run-nixbot-gitlab> gitlab # [  296.975520] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.805Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ef1b99f71d9cf18898aef080","created_at":"2026-07-04T10:54:43.275Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.290Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ef1b99f71d9cf18898aef080: done: 0.333656 sec","job_status":"done","queue_duration_s":0.181099,"scheduling_latency_s":0.181139,"redis_calls":11,"redis_duration_s":0.003752,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000954,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00211,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000688,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.271,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.271,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.225024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000667,"duration_s":0.333656,"completed_at":"2026-07-04T10:54:43.805Z","load_balancing_strategy":"primary","db_duration_s":0.11263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1744vm-test-run-nixbot-gitlab> gitlab # [  297.066307] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.807Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8c382271b8ba5c972dcf1872","created_at":"2026-07-04T10:54:43.297Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.310Z","job_size_bytes":89,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8c382271b8ba5c972dcf1872: start","job_status":"start","queue_duration_s":0.497625,"scheduling_latency_s":0.497658}1745vm-test-run-nixbot-gitlab> gitlab # [  297.090577] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:43.921Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8c382271b8ba5c972dcf1872","created_at":"2026-07-04T10:54:43.297Z","trace_propagation_headers":{"sentry-trace":"2def5a6fe9a04363b73284e1780700b0-83d59d1618e04acc","baggage":"sentry-trace_id=2def5a6fe9a04363b73284e1780700b0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8AQM52P0ZYDQ50YJ17XZ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:43.310Z","job_size_bytes":89,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8c382271b8ba5c972dcf1872: done: 0.113747 sec","job_status":"done","queue_duration_s":0.497625,"scheduling_latency_s":0.497658,"redis_calls":11,"redis_duration_s":0.004413,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000961,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00123778,"duration_s":0.113747,"completed_at":"2026-07-04T10:54:43.921Z","load_balancing_strategy":"primary","db_duration_s":0.045835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1746vm-test-run-nixbot-gitlab> gitlab # [  297.158511] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:43 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 25621747vm-test-run-nixbot-gitlab> gitlab # [  297.161836] setup-gitlab-start[3240]: + mkdir -p /tmp/test-flake1748vm-test-run-nixbot-gitlab> gitlab # [  297.163176] setup-gitlab-start[3240]: + cd /tmp/test-flake1749vm-test-run-nixbot-gitlab> gitlab # [  297.164445] setup-gitlab-start[3240]: + git init -b master1750vm-test-run-nixbot-gitlab> gitlab # [  297.170575] setup-gitlab-start[3668]: Initialized empty Git repository in /tmp/test-flake/.git/1751vm-test-run-nixbot-gitlab> gitlab # [  297.172289] setup-gitlab-start[3240]: + git config user.name test1752vm-test-run-nixbot-gitlab> gitlab # [  297.177791] setup-gitlab-start[3240]: + git config user.email test@example.com1753vm-test-run-nixbot-gitlab> gitlab # [  297.184285] setup-gitlab-start[3240]: + cat1754vm-test-run-nixbot-gitlab> gitlab # [  297.189788] setup-gitlab-start[3240]: + git add flake.nix1755vm-test-run-nixbot-gitlab> gitlab # [  297.194864] setup-gitlab-start[3240]: + git commit -m 'initial commit'1756vm-test-run-nixbot-gitlab> gitlab # [  297.214094] setup-gitlab-start[3673]: [master (root-commit) 6e7cce8] initial commit1757vm-test-run-nixbot-gitlab> gitlab # [  297.217554] setup-gitlab-start[3673]:  1 file changed, 10 insertions(+)1758vm-test-run-nixbot-gitlab> gitlab # [  297.220425] setup-gitlab-start[3673]:  create mode 100644 flake.nix1759vm-test-run-nixbot-gitlab> gitlab # [  297.223085] setup-gitlab-start[3240]: + git remote add origin http://oauth2:glpat-AP-PCMuJK8tOx16BPedHGW86MQp1OjEH.01.0w05btfck@gitlab/root/test-flake.git1760vm-test-run-nixbot-gitlab> gitlab # [  297.228948] setup-gitlab-start[3240]: + git push -u origin master1761vm-test-run-nixbot-gitlab> gitlab # [  297.342964] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 651762vm-test-run-nixbot-gitlab> gitlab # [  297.521783] gitaly[1383]: time="2026-07-04T10:54:44.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=6 command.system_time_ms=7 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DEG0E7M1EZCK4N3KG3W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:44.337" grpc.time_ms=16.046 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1763vm-test-run-nixbot-gitlab> gitlab # [  297.565684] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 1771764vm-test-run-nixbot-gitlab> gitlab # [  298.290678] gitaly[1383]: time="2026-07-04T10:54:45.121Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:55.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.107" grpc.time_ms=13.573 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765vm-test-run-nixbot-gitlab> gitlab # [  298.344183] gitaly[1383]: time="2026-07-04T10:54:45.174Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T10:55:15.050" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T10:54:45.151" grpc.time_ms=22.518 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1766vm-test-run-nixbot-gitlab> gitlab # [  298.388238] gitaly[1383]: time="2026-07-04T10:54:45.202Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA duration_ms=538ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/allowed"1767vm-test-run-nixbot-gitlab> gitlab # [  298.404925] gitaly[1383]: time="2026-07-04T10:54:45.236Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA duration_ms=33ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/pre_receive"1768vm-test-run-nixbot-gitlab> gitlab # [  298.414413] gitaly[1383]: time="2026-07-04T10:54:45.237Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:44.662" grpc.time_ms=574.473 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1769vm-test-run-nixbot-gitlab> gitlab # [  298.441774] gitaly[1383]: time="2026-07-04T10:54:45.273Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.272" grpc.time_ms=0.593 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1770vm-test-run-nixbot-gitlab> gitlab # [  298.491562] gitaly[1383]: time="2026-07-04T10:54:45.323Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.322" grpc.time_ms=0.565 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1771vm-test-run-nixbot-gitlab> gitlab # [  298.531300] gitaly[1383]: time="2026-07-04T10:54:45.362Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.362" grpc.time_ms=0.544 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1772vm-test-run-nixbot-gitlab> gitlab # [  298.636756] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:45.465Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2e47b8e38367b2074030c93a","created_at":"2026-07-04T10:54:45.453Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-6612d46627f3449c","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.462Z","job_size_bytes":185,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2e47b8e38367b2074030c93a: start","job_status":"start","queue_duration_s":0.002122,"scheduling_latency_s":0.002178}1773vm-test-run-nixbot-gitlab> gitlab # [  298.674183] gitaly[1383]: time="2026-07-04T10:54:45.503Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:55.299" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.489" grpc.time_ms=13.62 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774vm-test-run-nixbot-gitlab> gitlab # [  298.710167] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:45.519Z","message":"changes: \"0000000000000000000000000000000000000000 6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 refs/heads/master\\n\""}1775vm-test-run-nixbot-gitlab> gitlab # [  298.724007] gitaly[1383]: time="2026-07-04T10:54:45.555Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=24 command.system_time_ms=14 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T10:54:55.006" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.517" grpc.time_ms=38.375 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776vm-test-run-nixbot-gitlab> gitlab # [  298.752221] gitaly[1383]: time="2026-07-04T10:54:45.578Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA duration_ms=185ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/post_receive"1777vm-test-run-nixbot-gitlab> gitlab # [  298.757540] gitaly[1383]: time="2026-07-04T10:54:45.579Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.392" grpc.time_ms=186.682 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1778vm-test-run-nixbot-gitlab> gitlab # [  298.777087] gitaly[1383]: time="2026-07-04T10:54:45.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=9 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:54:55.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.571" grpc.time_ms=16.886 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779vm-test-run-nixbot-gitlab> gitlab # [  298.798774] gitaly[1383]: time="2026-07-04T10:54:45.591Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA diskcache=7c9a8fc2-c9f0-4fe3-8991-c44044b15d3e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:44.547" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1780vm-test-run-nixbot-gitlab> gitlab # [  298.814501] systemd[1]: Finished setup-gitlab.service.1781vm-test-run-nixbot-gitlab> gitlab # [  298.816115] gitlab-workhorse[1302]: time="2026-07-04T10:54:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521782vm-test-run-nixbot-gitlab> gitlab # [  298.818771] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:45 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 11781783vm-test-run-nixbot-gitlab> gitlab # [  298.821744] setup-gitlab-start[3677]: To http://gitlab/root/test-flake.git1784vm-test-run-nixbot-gitlab> gitlab # [  298.822959] setup-gitlab-start[3677]:  * [new branch]      master -> master1785vm-test-run-nixbot-gitlab> gitlab # [  298.824198] setup-gitlab-start[3677]: branch 'master' set up to track 'origin/master'.1786vm-test-run-nixbot-gitlab> gitlab # [  298.825582] gitaly[1383]: time="2026-07-04T10:54:45.591Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=276 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13211 command.oublock=32 command.real_time_ms=1036 command.spawn_token_fork_ms=1 command.system_time_ms=148 command.user_time_ms=128 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:44.547" grpc.time_ms=1043.345 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1787vm-test-run-nixbot-gitlab> gitlab # [  298.842183] systemd[1]: Starting nixbot CI service...1788vm-test-run-nixbot-gitlab> gitlab # [  298.843076] gitaly[1383]: time="2026-07-04T10:54:45.605Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:54:55.024" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.594" grpc.time_ms=10.643 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789vm-test-run-nixbot-gitlab> gitlab # [  298.857510] gitaly[1383]: time="2026-07-04T10:54:45.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T10:54:55.070" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.640" grpc.time_ms=9.586 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790vm-test-run-nixbot-gitlab> gitlab # [  298.871327] gitaly[1383]: time="2026-07-04T10:54:45.691Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.690" grpc.time_ms=0.607 pid=1383 span.kind=server system=grpc1791vm-test-run-nixbot-gitlab> gitlab # [  298.887430] gitaly[1383]: time="2026-07-04T10:54:45.719Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:45.718" grpc.time_ms=0.439 pid=1383 span.kind=server system=grpc1792vm-test-run-nixbot-gitlab> gitlab # [  298.902420] gitaly[1383]: time="2026-07-04T10:54:45.734Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA diskcache=dcbd1b6d-5588-4734-8a18-12e25f4d6b1c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T10:54:55.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.662" pid=1383 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1793vm-test-run-nixbot-gitlab> gitlab # [  298.921631] gitaly[1383]: time="2026-07-04T10:54:45.734Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=63 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5147 command.oublock=8 command.real_time_ms=66 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T10:54:55.092" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.662" grpc.time_ms=71.255 pid=1383 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1794vm-test-run-nixbot-gitlab> gitlab # [  298.943527] gitaly[1383]: time="2026-07-04T10:54:45.746Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-04T10:54:55.176" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.746" grpc.time_ms=0.159 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1795vm-test-run-nixbot-gitlab> gitlab # [  298.958612] gitaly[1383]: time="2026-07-04T10:54:45.761Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=861 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-04T10:54:55.179" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.750" grpc.time_ms=11.31 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1796vm-test-run-nixbot-gitlab> gitlab # [  299.025423] gitaly[1383]: time="2026-07-04T10:54:45.857Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-04T10:55:15.051" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:45.851" grpc.time_ms=5.274 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797vm-test-run-nixbot-gitlab> gitlab # [  299.062897] gitaly[1383]: time="2026-07-04T10:54:45.894Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T10:54:55.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:45.894" grpc.time_ms=0.211 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1798vm-test-run-nixbot-gitlab> gitlab # [  299.081817] gitaly[1383]: time="2026-07-04T10:54:45.905Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T10:54:55.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T10:54:45.898" grpc.time_ms=6.898 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1799vm-test-run-nixbot-gitlab> gitlab # [  299.102632] gitaly[1383]: time="2026-07-04T10:54:45.931Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:54:55.033" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.924" grpc.time_ms=7.055 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800vm-test-run-nixbot-gitlab> gitlab # [  299.121384] gitaly[1383]: time="2026-07-04T10:54:45.943Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:54:55.045" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:45.936" grpc.time_ms=6.844 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801vm-test-run-nixbot-gitlab> gitlab # [  299.140134] gitaly[1383]: time="2026-07-04T10:54:45.964Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T10:55:15.152" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:45.953" grpc.time_ms=10.994 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 300.08 seconds)1803vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service1804vm-test-run-nixbot-gitlab> gitlab # [  299.734325] gitaly[1383]: time="2026-07-04T10:54:46.564Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T10:54:56.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:46.562" grpc.time_ms=1.925 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805vm-test-run-nixbot-gitlab> gitlab # [  299.779912] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.608Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2e47b8e38367b2074030c93a","created_at":"2026-07-04T10:54:45.453Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-6612d46627f3449c","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.462Z","job_size_bytes":185,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2e47b8e38367b2074030c93a: done: 1.1433 sec","job_status":"done","queue_duration_s":0.002122,"scheduling_latency_s":0.002178,"gitaly_calls":13,"gitaly_duration_s":0.196668,"redis_calls":195,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.047498,"redis_read_bytes":1461,"redis_write_bytes":18936,"redis_cache_calls":14,"redis_cache_duration_s":0.004752,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.007555,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001463,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.0033,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.015676,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014752,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.503,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.533,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.073,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.653436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00112694,"exclusive_lock_hold_duration_s":0.04916374,"duration_s":1.1433,"completed_at":"2026-07-04T10:54:46.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.107176,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1806vm-test-run-nixbot-gitlab> gitlab # [  299.890620] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.610Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"bbe1cd9b518780694e8fc231","created_at":"2026-07-04T10:54:45.608Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.614Z","job_size_bytes":46,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bbe1cd9b518780694e8fc231: start","job_status":"start","queue_duration_s":0.996045,"scheduling_latency_s":0.996076}1807vm-test-run-nixbot-gitlab> gitlab # [  299.914787] gitaly[1383]: time="2026-07-04T10:54:46.686Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-04T16:54:46.676" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:46.677" grpc.time_ms=8.941 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808vm-test-run-nixbot-gitlab> gitlab # [  299.930708] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.741Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"bbe1cd9b518780694e8fc231","created_at":"2026-07-04T10:54:45.608Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.614Z","job_size_bytes":46,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-bbe1cd9b518780694e8fc231: done: 0.131574 sec","job_status":"done","queue_duration_s":0.996045,"scheduling_latency_s":0.996076,"gitaly_calls":2,"gitaly_duration_s":0.026984,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009291,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001731,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000652,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001888,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004098,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003428,"duration_s":0.131574,"completed_at":"2026-07-04T10:54:46.741Z","load_balancing_strategy":"primary","db_duration_s":0.018517,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1809vm-test-run-nixbot-gitlab> gitlab # [  299.997252] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.743Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"194834a3d2a552011f628bd3","created_at":"2026-07-04T10:54:45.631Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.636Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-194834a3d2a552011f628bd3: start","job_status":"start","queue_duration_s":1.106698,"scheduling_latency_s":1.106718}1810vm-test-run-nixbot-gitlab> gitlab # [  300.017003] gitaly[1383]: time="2026-07-04T10:54:46.710Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T10:55:16.096" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:46.697" grpc.time_ms=12.865 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811vm-test-run-nixbot-gitlab> gitlab # [  300.032670] gitaly[1383]: time="2026-07-04T10:54:46.792Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA diskcache=5565153f-0b26-4968-b20f-3f72c95d62bc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T16:54:46.767" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:46.767" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812vm-test-run-nixbot-gitlab> gitlab # [  300.044089] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.827Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"194834a3d2a552011f628bd3","created_at":"2026-07-04T10:54:45.631Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.636Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-194834a3d2a552011f628bd3: done: 0.083961 sec","job_status":"done","queue_duration_s":1.106698,"scheduling_latency_s":1.106718,"gitaly_calls":1,"gitaly_duration_s":0.027106,"redis_calls":15,"redis_duration_s":0.0038729999999999997,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000657,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06035434,"duration_s":0.083961,"completed_at":"2026-07-04T10:54:46.827Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1813vm-test-run-nixbot-gitlab> gitlab # [  300.093168] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.829Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8e4f04b6291471c94ac02fd4","created_at":"2026-07-04T10:54:45.742Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.744Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e4f04b6291471c94ac02fd4: start","job_status":"start","queue_duration_s":1.084895,"scheduling_latency_s":1.084914}1814vm-test-run-nixbot-gitlab> gitlab # [  300.109730] nixbot[3780]: {"ts": "2026-07-04T10:54:46.866Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1815vm-test-run-nixbot-gitlab> gitlab # [  300.112062] nixbot[3780]: {"ts": "2026-07-04T10:54:46.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1816vm-test-run-nixbot-gitlab> gitlab # [  300.114152] nixbot[3780]: {"ts": "2026-07-04T10:54:46.902Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1817vm-test-run-nixbot-gitlab> gitlab # [  300.116367] nixbot[3780]: {"ts": "2026-07-04T10:54:46.906Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1818vm-test-run-nixbot-gitlab> gitlab # [  300.118593] nixbot[3780]: {"ts": "2026-07-04T10:54:46.910Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1819vm-test-run-nixbot-gitlab> gitlab # [  300.120778] nixbot[3780]: {"ts": "2026-07-04T10:54:46.915Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1820vm-test-run-nixbot-gitlab> gitlab # [  300.122928] nixbot[3780]: {"ts": "2026-07-04T10:54:46.919Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1821vm-test-run-nixbot-gitlab> gitlab # [  300.125136] nixbot[3780]: {"ts": "2026-07-04T10:54:46.920Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1822vm-test-run-nixbot-gitlab> gitlab # [  300.127220] nixbot[3780]: {"ts": "2026-07-04T10:54:46.925Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1823vm-test-run-nixbot-gitlab> gitlab # [  300.129338] nixbot[3780]: {"ts": "2026-07-04T10:54:46.927Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1824vm-test-run-nixbot-gitlab> gitlab # [  300.131400] nixbot[3780]: {"ts": "2026-07-04T10:54:46.932Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1825vm-test-run-nixbot-gitlab> gitlab # [  300.133496] nixbot[3780]: {"ts": "2026-07-04T10:54:46.937Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1826vm-test-run-nixbot-gitlab> gitlab # [  300.135645] gitaly[1383]: time="2026-07-04T10:54:46.793Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1176 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T16:54:46.767" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:46.767" grpc.time_ms=24.879 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827vm-test-run-nixbot-gitlab> gitlab # [  300.152058] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.884Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8e4f04b6291471c94ac02fd4","created_at":"2026-07-04T10:54:45.742Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:45.744Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8e4f04b6291471c94ac02fd4: done: 0.055092 sec","job_status":"done","queue_duration_s":1.084895,"scheduling_latency_s":1.084914,"gitaly_calls":1,"gitaly_duration_s":0.018484,"redis_calls":16,"redis_duration_s":0.005541,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.00039,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002963,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055092,"completed_at":"2026-07-04T10:54:46.884Z","load_balancing_strategy":"primary","db_duration_s":0.002506,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1828vm-test-run-nixbot-gitlab> gitlab # [  300.201579] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:46.886Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3bf1ca277fce14ccda165d2a","created_at":"2026-07-04T10:54:46.516Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82be4388de894e11d2af432172ca53b2241305f9ef640503c6b32379b9b4318a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:46.524Z","job_size_bytes":467,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3bf1ca277fce14ccda165d2a: start","job_status":"start","queue_duration_s":0.36165,"scheduling_latency_s":0.361674}1829vm-test-run-nixbot-gitlab> gitlab # [  300.221275] nixbot[3780]: {"ts": "2026-07-04T10:54:46.940Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1830vm-test-run-nixbot-gitlab> gitlab # [  300.223449] nixbot[3780]: {"ts": "2026-07-04T10:54:46.942Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1831vm-test-run-nixbot-gitlab> gitlab # [  300.225790] nixbot[3780]: {"ts": "2026-07-04T10:54:46.946Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1832vm-test-run-nixbot-gitlab> gitlab # [  300.227959] nixbot[3780]: {"ts": "2026-07-04T10:54:46.948Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1833vm-test-run-nixbot-gitlab> gitlab # [  300.230182] nixbot[3780]: {"ts": "2026-07-04T10:54:46.949Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1834vm-test-run-nixbot-gitlab> gitlab # [  300.232321] nixbot[3780]: {"ts": "2026-07-04T10:54:46.951Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1835vm-test-run-nixbot-gitlab> gitlab # [  300.234347] nixbot[3780]: {"ts": "2026-07-04T10:54:46.954Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1836vm-test-run-nixbot-gitlab> gitlab # [  300.236462] nixbot[3780]: {"ts": "2026-07-04T10:54:46.956Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1837vm-test-run-nixbot-gitlab> gitlab # [  300.238596] gitaly[1383]: time="2026-07-04T10:54:46.873Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=593 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-04T10:55:16.256" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:46.857" grpc.time_ms=16.085 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838vm-test-run-nixbot-gitlab> gitlab # [  300.251999] gitaly[1383]: time="2026-07-04T10:54:47.037Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:17.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:47.036" grpc.time_ms=0.742 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839vm-test-run-nixbot-gitlab> gitlab # [  300.265638] gitaly[1383]: time="2026-07-04T10:54:47.052Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T10:54:57.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:47.040" grpc.time_ms=11.056 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840vm-test-run-nixbot-gitlab> gitlab # [  300.398140] gitaly[1383]: time="2026-07-04T10:54:47.229Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T10:55:17.227" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:47.227" grpc.time_ms=1.741 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841vm-test-run-nixbot-gitlab> gitlab # [  300.435499] gitaly[1383]: time="2026-07-04T10:54:47.239Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=158 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T10:55:17.233" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:47.234" grpc.time_ms=4.882 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842vm-test-run-nixbot-gitlab> gitlab # [  300.545199] gitaly[1383]: time="2026-07-04T10:54:47.376Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:54:57.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:47.358" grpc.time_ms=17.752 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843vm-test-run-nixbot-gitlab> gitlab # [  300.559463] nixbot[3780]: INFO:     Started server process [3780]1844vm-test-run-nixbot-gitlab> gitlab # [  300.560813] nixbot[3780]: INFO:     Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1845vm-test-run-nixbot-gitlab> gitlab # [  300.562529] nixbot[3780]: INFO:     ::ffff:127.0.0.1:39054 - "GET /health HTTP/1.1" 200 OK1846vm-test-run-nixbot-gitlab> gitlab # [  300.568945] systemd[1]: Started nixbot CI service.1847vm-test-run-nixbot-gitlab> gitlab # [  300.571129] systemd[1]: Reached target Multi-User System.1848vm-test-run-nixbot-gitlab> gitlab # [  300.576453] systemd[1]: Startup finished in 815ms (kernel) + 5.029s (initrd) + 4min 54.731s (userspace) = 5min 576ms.1849vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 1.12 seconds)1850vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1851vm-test-run-nixbot-gitlab> gitlab # [  300.787170] nixbot[3780]: INFO:     ::ffff:127.0.0.1:39058 - "GET /health HTTP/1.1" 200 OK1852vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds)1853vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered1854vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1855vm-test-run-nixbot-gitlab> gitlab # [  300.851551] gitaly[1383]: time="2026-07-04T10:54:47.680Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=884 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8GK01W5A5B7BGB49DGQ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T10:55:17.054" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:47.654" grpc.time_ms=25.731 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856vm-test-run-nixbot-gitlab> gitlab # [  300.909497] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:47 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 3481857vm-test-run-nixbot-gitlab> gitlab # [  300.913743] nixbot[3780]: {"ts": "2026-07-04T10:54:47.742Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1858vm-test-run-nixbot-gitlab> gitlab # [  300.936186] nixbot[3780]: {"ts": "2026-07-04T10:54:47.767Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1859vm-test-run-nixbot-gitlab> gitlab # [  300.951161] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 1071860vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1861vm-test-run-nixbot-gitlab> []1862vm-test-run-nixbot-gitlab> gitlab # [  301.034426] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 911863vm-test-run-nixbot-gitlab> gitlab # [  301.039038] nixbot[3780]: {"ts": "2026-07-04T10:54:47.868Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1864vm-test-run-nixbot-gitlab> gitlab # [  301.043686] nixbot[3780]: {"ts": "2026-07-04T10:54:47.871Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1865vm-test-run-nixbot-gitlab> gitlab # [  301.175375] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 1311866vm-test-run-nixbot-gitlab> gitlab # [  301.184390] nixbot[3780]: {"ts": "2026-07-04T10:54:48.007Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1867vm-test-run-nixbot-gitlab> gitlab # [  301.432567] gitaly[1383]: time="2026-07-04T10:54:48.264Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=883 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T10:55:18.238" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:48.238" grpc.time_ms=25.394 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868vm-test-run-nixbot-gitlab> gitlab # [  301.477031] gitaly[1383]: time="2026-07-04T10:54:48.283Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8DNXMTAJ6S5JJ16N8ZKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T10:55:18.269" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:48.269" grpc.time_ms=13.811 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869vm-test-run-nixbot-gitlab> gitlab # [  301.509365] sidekiq-cluster[2431]: {"severity":"WARN","time":"2026-07-04T10:54:48.340Z","class":"CreatePipelineWorker","correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"6e7cce8b98ba58daeec84886a41b1e5ddf3252a5","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6e7cce8b98ba58daeec84886a41b1e5ddf3252a5","ref":"refs/heads/master"},"retry":0}1870vm-test-run-nixbot-gitlab> gitlab # [  301.536867] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:48.367Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3bf1ca277fce14ccda165d2a","created_at":"2026-07-04T10:54:46.516Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-2cfd93b8cce24cd6","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82be4388de894e11d2af432172ca53b2241305f9ef640503c6b32379b9b4318a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:46.524Z","job_size_bytes":467,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3bf1ca277fce14ccda165d2a: done: 1.481723 sec","job_status":"done","queue_duration_s":0.36165,"scheduling_latency_s":0.361674,"gitaly_calls":7,"gitaly_duration_s":0.101572,"redis_calls":53,"redis_duration_s":0.041314,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.002174,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.00577,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.007117,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.007371,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.222729,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056642,"duration_s":1.481723,"completed_at":"2026-07-04T10:54:48.367Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.115987,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1871vm-test-run-nixbot-gitlab> gitlab # [  301.624173] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:48.370Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"f0bd1eec97035df3c58266e2","created_at":"2026-07-04T10:54:46.722Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-cd27911d19bd42fe","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:46.728Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-f0bd1eec97035df3c58266e2: start","job_status":"start","queue_duration_s":1.641586,"scheduling_latency_s":1.641618}1872vm-test-run-nixbot-gitlab> gitlab # [  301.643312] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:48.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"f0bd1eec97035df3c58266e2","created_at":"2026-07-04T10:54:46.722Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-cd27911d19bd42fe","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:46.728Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-f0bd1eec97035df3c58266e2: done: 0.067058 sec","job_status":"done","queue_duration_s":1.641586,"scheduling_latency_s":1.641618,"redis_calls":19,"redis_duration_s":0.005279,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001147,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000714,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002412,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0110087,"duration_s":0.067058,"completed_at":"2026-07-04T10:54:48.437Z","load_balancing_strategy":"primary","db_duration_s":0.01157,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873vm-test-run-nixbot-gitlab> gitlab # [  301.691559] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:48.439Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"957a1e675c1e3ac73a274ef3","created_at":"2026-07-04T10:54:48.418Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-87b1cdd953774d4c","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:48.426Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-957a1e675c1e3ac73a274ef3: start","job_status":"start","queue_duration_s":0.0128,"scheduling_latency_s":0.012823}1874vm-test-run-nixbot-gitlab> gitlab # [  301.720416] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:48.551Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"957a1e675c1e3ac73a274ef3","created_at":"2026-07-04T10:54:48.418Z","trace_propagation_headers":{"sentry-trace":"8e29b0f6fa65435eaf10c5d851cd671f-87b1cdd953774d4c","baggage":"sentry-trace_id=8e29b0f6fa65435eaf10c5d851cd671f,sentry-release=Unknown"},"correlation_id":"01KWPC8DNXMTAJ6S5JJ16N8ZKA","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:48.426Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-957a1e675c1e3ac73a274ef3: done: 0.112328 sec","job_status":"done","queue_duration_s":0.0128,"scheduling_latency_s":0.012823,"redis_calls":11,"redis_duration_s":0.00231,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000585,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001327,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025784,"duration_s":0.112328,"completed_at":"2026-07-04T10:54:48.551Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1875vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1876vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.16 seconds)1877vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-04T10:54:47.991Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]1878vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.33 seconds)1879vm-test-run-nixbot-gitlab> subtest: push delivers the webhook1880vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1881vm-test-run-nixbot-gitlab> gitlab # [  302.115548] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 1011882vm-test-run-nixbot-gitlab> gitlab # [  302.262588] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 581883vm-test-run-nixbot-gitlab> gitlab # [  302.379783] gitaly[1383]: time="2026-07-04T10:54:49.210Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8J87C40D5JX38Q9SRFAQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:49.196" grpc.time_ms=13.848 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1884vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done.1885vm-test-run-nixbot-gitlab> gitlab # [  302.400684] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:49 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 1171886vm-test-run-nixbot-gitlab> gitlab # Counting objects:  20% (1/5)Counting objects:  40% (2/5)Counting objects:  60% (3/5)Counting objects:  80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1887vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads1888vm-test-run-nixbot-gitlab> gitlab # Compressing objects:  50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1889vm-test-run-nixbot-gitlab> gitlab # Writing objects:  33% (1/3)Writing objects:  66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 91.00 KiB/s, done.1890vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1891vm-test-run-nixbot-gitlab> gitlab # [  302.847221] gitaly[1383]: time="2026-07-04T10:54:49.678Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T10:54:59.156" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:49.667" grpc.time_ms=11.108 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892vm-test-run-nixbot-gitlab> gitlab # [  302.970071] gitaly[1383]: time="2026-07-04T10:54:49.801Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-04T10:55:19.183" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-04T10:54:49.784" grpc.time_ms=16.968 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893vm-test-run-nixbot-gitlab> gitlab # [  303.057185] gitaly[1383]: time="2026-07-04T10:54:49.888Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G duration_ms=454ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/allowed"1894vm-test-run-nixbot-gitlab> gitlab # [  303.115039] gitaly[1383]: time="2026-07-04T10:54:49.946Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G duration_ms=57ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/pre_receive"1895vm-test-run-nixbot-gitlab> gitlab # [  303.127969] gitaly[1383]: time="2026-07-04T10:54:49.947Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5318 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:49.433" grpc.time_ms=513.368 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1896vm-test-run-nixbot-gitlab> gitlab # [  303.165420] gitaly[1383]: time="2026-07-04T10:54:49.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:49.995" grpc.time_ms=0.858 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1897vm-test-run-nixbot-gitlab> gitlab # [  303.203421] gitaly[1383]: time="2026-07-04T10:54:50.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:50.034" grpc.time_ms=0.864 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1898vm-test-run-nixbot-gitlab> gitlab # [  303.267441] gitaly[1383]: time="2026-07-04T10:54:50.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:50.098" grpc.time_ms=0.65 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1899vm-test-run-nixbot-gitlab> gitlab # [  303.393915] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.223Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 7f17f820668d42138a4ec71e7024cc120878ae6f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a5cb7cb2d1a46a6cbe69d802","created_at":"2026-07-04T10:54:50.219Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-7142f8b4cd384914","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.222Z","job_size_bytes":185,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a5cb7cb2d1a46a6cbe69d802: start","job_status":"start","queue_duration_s":0.00176,"scheduling_latency_s":0.001805}1900vm-test-run-nixbot-gitlab> gitlab # [  303.441902] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.244Z","message":"changes: \"6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 7f17f820668d42138a4ec71e7024cc120878ae6f refs/heads/master\\n\""}1901vm-test-run-nixbot-gitlab> gitlab # [  303.457676] gitaly[1383]: time="2026-07-04T10:54:50.288Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=10 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:55:00.082" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:50.272" grpc.time_ms=16.005 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902vm-test-run-nixbot-gitlab> gitlab # [  303.475802] gitaly[1383]: time="2026-07-04T10:54:50.289Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=7 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:00.234" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:50.265" grpc.time_ms=24.231 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903vm-test-run-nixbot-gitlab> gitlab # [  303.496804] gitaly[1383]: time="2026-07-04T10:54:50.303Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-04T10:55:00.264" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:50.294" grpc.time_ms=9.226 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904vm-test-run-nixbot-gitlab> gitlab # [  303.512631] gitaly[1383]: time="2026-07-04T10:54:50.333Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G duration_ms=155ns method=POST pid=1383 status=200 url="http://unix/api/v4/internal/post_receive"1905vm-test-run-nixbot-gitlab> gitlab # [  303.515656] gitaly[1383]: time="2026-07-04T10:54:50.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4812 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:50.174" grpc.time_ms=158.474 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1906vm-test-run-nixbot-gitlab> gitlab # [  303.526921] gitaly[1383]: time="2026-07-04T10:54:50.352Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-04T10:55:00.010" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:50.340" grpc.time_ms=11.523 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907vm-test-run-nixbot-gitlab> gitlab # [  303.541813] gitaly[1383]: time="2026-07-04T10:54:50.368Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=599 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-04T10:55:20.355" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.356" grpc.time_ms=11.775 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908vm-test-run-nixbot-gitlab> gitlab # [  303.558355] gitaly[1383]: time="2026-07-04T10:54:50.377Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G diskcache=b0246ae2-fb39-4eeb-b596-e6fbc590c9b8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:49.338" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1909vm-test-run-nixbot-gitlab> gitlab # [  303.570140] gitlab-workhorse[1302]: time="2026-07-04T10:54:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521910vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git1911vm-test-run-nixbot-gitlab> gitlab # [  303.572101] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:50 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 11261912vm-test-run-nixbot-gitlab> gitlab #    6e7cce8..7f17f82  [  303.574880] gitaly[1383]: time="2026-07-04T10:54:50.377Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=250 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=12977 command.oublock=32 command.real_time_ms=1000 command.spawn_token_fork_ms=3 command.system_time_ms=177 command.user_time_ms=73 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:49.338" grpc.time_ms=1038.715 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1913vm-test-run-nixbot-gitlab> gitlab # master -> master[  303.589421] gitaly[1383]: time="2026-07-04T10:54:50.387Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-04T10:55:00.031" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-04T10:54:50.382" grpc.time_ms=5.538 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914vm-test-run-nixbot-gitlab> gitlab # 1915vm-test-run-nixbot-gitlab> gitlab # [  303.603890] gitaly[1383]: time="2026-07-04T10:54:50.408Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:00.050" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:50.400" grpc.time_ms=7.685 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.51 seconds)1917vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master1918vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1919vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f'1920vm-test-run-nixbot-gitlab> gitlab # [  303.664698] nixbot[3780]: INFO:     ::ffff:127.0.0.1:47294 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 200 OK1921vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f', in 0.03 seconds)1922vm-test-run-nixbot-gitlab> []1923vm-test-run-nixbot-gitlab> gitlab # [  303.782619] gitaly[1383]: time="2026-07-04T10:54:50.614Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-04T10:55:00.261" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.612" grpc.time_ms=1.494 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924vm-test-run-nixbot-gitlab> gitlab # [  303.823283] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.633Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","6e7cce8b98ba58daeec84886a41b1e5ddf3252a5 7f17f820668d42138a4ec71e7024cc120878ae6f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a5cb7cb2d1a46a6cbe69d802","created_at":"2026-07-04T10:54:50.219Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-7142f8b4cd384914","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.222Z","job_size_bytes":185,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a5cb7cb2d1a46a6cbe69d802: done: 0.409959 sec","job_status":"done","queue_duration_s":0.00176,"scheduling_latency_s":0.001805,"gitaly_calls":7,"gitaly_duration_s":0.08818,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.027511,"redis_read_bytes":350,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.002104,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.008748,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003156,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006173,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006677,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.248861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00067322,"exclusive_lock_hold_duration_s":0.03400818,"duration_s":0.409959,"completed_at":"2026-07-04T10:54:50.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.061241,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1925vm-test-run-nixbot-gitlab> gitlab # [  303.906047] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.635Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"37c50502acd9a78a47b699ec","created_at":"2026-07-04T10:54:50.307Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.310Z","job_size_bytes":46,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-37c50502acd9a78a47b699ec: start","job_status":"start","queue_duration_s":0.324959,"scheduling_latency_s":0.324983}1926vm-test-run-nixbot-gitlab> gitlab # [  303.922507] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.658Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"37c50502acd9a78a47b699ec","created_at":"2026-07-04T10:54:50.307Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.310Z","job_size_bytes":46,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-37c50502acd9a78a47b699ec: done: 0.023351 sec","job_status":"done","queue_duration_s":0.324959,"scheduling_latency_s":0.324983,"redis_calls":9,"redis_duration_s":0.003096,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000361,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023351,"completed_at":"2026-07-04T10:54:50.658Z","load_balancing_strategy":"primary","db_duration_s":0.00293,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1927vm-test-run-nixbot-gitlab> gitlab # [  303.968004] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.660Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"17a7a090fe5e6a3b97d12621","created_at":"2026-07-04T10:54:50.327Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.329Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-17a7a090fe5e6a3b97d12621: start","job_status":"start","queue_duration_s":0.330725,"scheduling_latency_s":0.330746}1928vm-test-run-nixbot-gitlab> gitlab # [  303.984715] gitaly[1383]: time="2026-07-04T10:54:50.728Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G diskcache=87e141e3-4f9e-43f3-b27d-5df7066b7ffa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T16:54:50.679" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.680" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929vm-test-run-nixbot-gitlab> gitlab # [  303.994760] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.761Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"17a7a090fe5e6a3b97d12621","created_at":"2026-07-04T10:54:50.327Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.329Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-17a7a090fe5e6a3b97d12621: done: 0.101154 sec","job_status":"done","queue_duration_s":0.330725,"scheduling_latency_s":0.330746,"gitaly_calls":1,"gitaly_duration_s":0.051154,"redis_calls":12,"redis_duration_s":0.003762,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00071,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001549,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07637772,"duration_s":0.101154,"completed_at":"2026-07-04T10:54:50.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1930vm-test-run-nixbot-gitlab> gitlab # [  304.041864] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:50.764Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"4ede263820be891f6a148c5a","created_at":"2026-07-04T10:54:50.457Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:77305b8f6490938a73888ac051eae70a5666e13611f21e4f57cf874fef4c065b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.459Z","job_size_bytes":467,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-4ede263820be891f6a148c5a: start","job_status":"start","queue_duration_s":0.304748,"scheduling_latency_s":0.304785}1931vm-test-run-nixbot-gitlab> gitlab # [  304.059744] gitaly[1383]: time="2026-07-04T10:54:50.729Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=26 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1511 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=9 command.system_time_ms=20 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-04T16:54:50.679" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.680" grpc.time_ms=47.733 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932vm-test-run-nixbot-gitlab> gitlab # [  304.075702] gitaly[1383]: time="2026-07-04T10:54:50.809Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:20.806" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.807" grpc.time_ms=1.12 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933vm-test-run-nixbot-gitlab> gitlab # [  304.089058] gitaly[1383]: time="2026-07-04T10:54:50.827Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=576 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T10:55:00.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:50.813" grpc.time_ms=13.638 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934vm-test-run-nixbot-gitlab> gitlab # [  304.102119] gitaly[1383]: time="2026-07-04T10:54:50.883Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T10:55:20.079" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.881" grpc.time_ms=1.642 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935vm-test-run-nixbot-gitlab> gitlab # [  304.115233] gitaly[1383]: time="2026-07-04T10:54:50.895Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=275 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-04T10:55:20.088" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:50.888" grpc.time_ms=6.321 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936vm-test-run-nixbot-gitlab> gitlab # [  304.137267] gitaly[1383]: time="2026-07-04T10:54:50.968Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:00.142" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:50.953" grpc.time_ms=15.282 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937vm-test-run-nixbot-gitlab> gitlab # [  304.273227] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 30901938vm-test-run-nixbot-gitlab> gitlab # [  304.280988] nixbot[3780]: {"ts": "2026-07-04T10:54:51.106Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1939vm-test-run-nixbot-gitlab> gitlab # [  304.389422] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 991940vm-test-run-nixbot-gitlab> gitlab # [  304.396370] nixbot[3780]: {"ts": "2026-07-04T10:54:51.222Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1941vm-test-run-nixbot-gitlab> gitlab # [  304.455714] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 611942vm-test-run-nixbot-gitlab> gitlab # [  304.458121] nixbot[3780]: {"ts": "2026-07-04T10:54:51.289Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1943vm-test-run-nixbot-gitlab> gitlab # [  304.540652] gitaly[1383]: time="2026-07-04T10:54:51.371Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MD3TGBGMCT4KG78W3W7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-04T10:55:21.056" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=255 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:51.357" grpc.time_ms=14.241 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944vm-test-run-nixbot-gitlab> gitlab # [  304.571507] gitaly[1383]: time="2026-07-04T10:54:51.403Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MD3TGBGMCT4KG78W3W7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:21.101" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:51.401" grpc.time_ms=0.862 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945vm-test-run-nixbot-gitlab> gitlab # [  304.595710] gitaly[1383]: time="2026-07-04T10:54:51.427Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MD3TGBGMCT4KG78W3W7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-04T10:55:01.219" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:51.410" grpc.time_ms=16.885 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f'1947vm-test-run-nixbot-gitlab> gitlab # [  304.677709] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 771 "" "python-httpx/0.28.1" 2081948vm-test-run-nixbot-gitlab> gitlab # [  304.682499] nixbot[3780]: {"ts": "2026-07-04T10:54:51.510Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1949vm-test-run-nixbot-gitlab> gitlab # [  304.718062] nixbot[3780]: INFO:     ::ffff:127.0.0.1:47298 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 200 OK1950vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f', in 0.05 seconds)1951vm-test-run-nixbot-gitlab> []1952vm-test-run-nixbot-gitlab> gitlab # [  304.808505] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1191953vm-test-run-nixbot-gitlab> gitlab # [  304.816468] nixbot[3780]: {"ts": "2026-07-04T10:54:51.640Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1954vm-test-run-nixbot-gitlab> gitlab # [  304.824937] nixbot[3780]: {"ts": "2026-07-04T10:54:51.646Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1955vm-test-run-nixbot-gitlab> gitlab # [  304.904689] gitaly[1383]: time="2026-07-04T10:54:51.735Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-04T10:55:21.021" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:51.722" grpc.time_ms=13.519 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1956vm-test-run-nixbot-gitlab> gitlab # [  304.947604] gitaly[1383]: time="2026-07-04T10:54:51.758Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8JD2G1YWDKM7B4FAH76G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-04T10:55:21.038" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:51.738" grpc.time_ms=19.58 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957vm-test-run-nixbot-gitlab> gitlab # [  304.971983] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 721958vm-test-run-nixbot-gitlab> gitlab # [  304.975194] gitaly[1383]: time="2026-07-04T10:54:51.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8MSS1WEAZPYFAME733YM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.769" grpc.time_ms=8.103 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959vm-test-run-nixbot-gitlab> gitlab # [  304.997034] sidekiq-cluster[2431]: {"severity":"WARN","time":"2026-07-04T10:54:51.804Z","class":"CreatePipelineWorker","correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"6e7cce8b98ba58daeec84886a41b1e5ddf3252a5","after":"7f17f820668d42138a4ec71e7024cc120878ae6f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7f17f820668d42138a4ec71e7024cc120878ae6f","ref":"refs/heads/master"},"retry":0}1960vm-test-run-nixbot-gitlab> gitlab # [  305.008484] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:51.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"4ede263820be891f6a148c5a","created_at":"2026-07-04T10:54:50.457Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:77305b8f6490938a73888ac051eae70a5666e13611f21e4f57cf874fef4c065b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.459Z","job_size_bytes":467,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-4ede263820be891f6a148c5a: done: 1.052196 sec","job_status":"done","queue_duration_s":0.304748,"scheduling_latency_s":0.304785,"gitaly_calls":7,"gitaly_duration_s":0.115762,"redis_calls":33,"redis_duration_s":0.010307,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001312,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001489,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003926,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00358,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.889726,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.052196,"completed_at":"2026-07-04T10:54:51.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043518,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1961vm-test-run-nixbot-gitlab> gitlab # [  305.070266] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:51.819Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"26acfe7a-1999-427e-8feb-e03a3c7aeecd\"}"],"class":"WebHookWorker","jid":"7a95da552002e93e64ae12d2","created_at":"2026-07-04T10:54:50.590Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2ef67d863c94f700e615ff8b3bc28313f07d97f05c9120b6bcce399797ff7db","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.597Z","job_size_bytes":1749,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7a95da552002e93e64ae12d2: start","job_status":"start","queue_duration_s":1.221698,"scheduling_latency_s":1.221738}1962vm-test-run-nixbot-gitlab> gitlab # [  305.088193] gitaly[1383]: time="2026-07-04T10:54:51.877Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MW7K6ZRSA94ZFJQ55XY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.858" pid=1383 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc1963vm-test-run-nixbot-gitlab> gitlab # [  305.098676] gitlab-workhorse[1302]: time="2026-07-04T10:54:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1491964vm-test-run-nixbot-gitlab> gitlab # [  305.100715] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 941965vm-test-run-nixbot-gitlab> gitlab # [  305.102669] gitaly[1383]: time="2026-07-04T10:54:51.877Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MW7K6ZRSA94ZFJQ55XY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.858" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1966vm-test-run-nixbot-gitlab> gitlab # [  305.116791] gitaly[1383]: time="2026-07-04T10:54:51.877Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MW7K6ZRSA94ZFJQ55XY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.858" grpc.time_ms=19.31 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1967vm-test-run-nixbot-gitlab> gitlab # [  305.132791] nixbot[3780]: INFO:     ::ffff:127.0.0.1:47304 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1968vm-test-run-nixbot-gitlab> gitlab # [  305.134088] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:51.959Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"26acfe7a-1999-427e-8feb-e03a3c7aeecd\"}"],"class":"WebHookWorker","jid":"7a95da552002e93e64ae12d2","created_at":"2026-07-04T10:54:50.590Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-176d2aabeb0d442c","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2ef67d863c94f700e615ff8b3bc28313f07d97f05c9120b6bcce399797ff7db","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:50.597Z","job_size_bytes":1749,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-7a95da552002e93e64ae12d2: done: 0.140689 sec","job_status":"done","queue_duration_s":1.221698,"scheduling_latency_s":1.221738,"redis_calls":19,"redis_duration_s":0.003629,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000688,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000316,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.015980819999981577,"cpu_s":0.1008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031148,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.140689,"completed_at":"2026-07-04T10:54:51.959Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969vm-test-run-nixbot-gitlab> gitlab # [  305.183319] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:51.961Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7a95da552002e93e64ae12d2"],"class":"WebHooks::LogExecutionWorker","jid":"f9643fb2ac97bf7228bbee93","created_at":"2026-07-04T10:54:51.945Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-291200e8bde34893","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b22186a907a6d12983dd5285a6ae9d5471198cdcf6c6185473b5fe7b600a8a74","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:51.951Z","job_size_bytes":2507,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f9643fb2ac97bf7228bbee93: start","job_status":"start","queue_duration_s":0.009572,"scheduling_latency_s":0.009597}1970vm-test-run-nixbot-gitlab> gitlab # [  305.200498] gitlab-workhorse[1302]: time="2026-07-04T10:54:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=5971971vm-test-run-nixbot-gitlab> gitlab # [  305.202334] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 597 "" "git/2.54.0" 681972vm-test-run-nixbot-gitlab> gitlab # [  305.205301] gitaly[1383]: time="2026-07-04T10:54:51.948Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MZ8XFTRN4QG3JHEQJ4W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.933" pid=1383 remote_ip="2001:db8:1::1" request_sha=7b3af1f25a8ac8dd0bc0e031b6f035a36b1588b7 response_bytes=597 span.kind=server system=grpc1973vm-test-run-nixbot-gitlab> gitlab # [  305.215761] gitaly[1383]: time="2026-07-04T10:54:51.948Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MZ8XFTRN4QG3JHEQJ4W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.933" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1974vm-test-run-nixbot-gitlab> gitlab # [  305.229732] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 591975vm-test-run-nixbot-gitlab> gitlab # [  305.231868] gitaly[1383]: time="2026-07-04T10:54:51.948Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8MZ8XFTRN4QG3JHEQJ4W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:51.933" grpc.time_ms=14.772 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976vm-test-run-nixbot-gitlab> gitlab # [  305.246959] gitaly[1383]: time="2026-07-04T10:54:52.039Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N2BM1923Q6P303SGVK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:52.028" grpc.time_ms=10.875 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1977vm-test-run-nixbot-gitlab> gitlab # [  305.272595] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.103Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","7a95da552002e93e64ae12d2"],"class":"WebHooks::LogExecutionWorker","jid":"f9643fb2ac97bf7228bbee93","created_at":"2026-07-04T10:54:51.945Z","trace_propagation_headers":{"sentry-trace":"23d41541f45141818419476e38e0b8bf-291200e8bde34893","baggage":"sentry-trace_id=23d41541f45141818419476e38e0b8bf,sentry-release=Unknown"},"correlation_id":"01KWPC8JD2G1YWDKM7B4FAH76G","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b22186a907a6d12983dd5285a6ae9d5471198cdcf6c6185473b5fe7b600a8a74","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:51.951Z","job_size_bytes":2507,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f9643fb2ac97bf7228bbee93: done: 0.142334 sec","job_status":"done","queue_duration_s":0.009572,"scheduling_latency_s":0.009597,"redis_calls":10,"redis_duration_s":0.0020109999999999998,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.034,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.034,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028764,"duration_s":0.142334,"completed_at":"2026-07-04T10:54:52.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978vm-test-run-nixbot-gitlab> gitlab # [  305.318525] gitaly[1383]: time="2026-07-04T10:54:52.127Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N497C1AKEMD9WZWJHHP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:52.119" pid=1383 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc1979vm-test-run-nixbot-gitlab> gitlab # [  305.329149] gitlab-workhorse[1302]: time="2026-07-04T10:54:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671980vm-test-run-nixbot-gitlab> gitlab # [  305.331018] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 861981vm-test-run-nixbot-gitlab> gitlab # [  305.333000] gitaly[1383]: time="2026-07-04T10:54:52.127Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N497C1AKEMD9WZWJHHP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:52.119" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1982vm-test-run-nixbot-gitlab> gitlab # [  305.346393] gitaly[1383]: time="2026-07-04T10:54:52.127Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N497C1AKEMD9WZWJHHP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-04T10:54:52.119" grpc.time_ms=8.248 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc1983vm-test-run-nixbot-gitlab> gitlab # [  305.380279] gitaly[1383]: time="2026-07-04T10:54:52.211Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:22.110" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:52.210" grpc.time_ms=0.771 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984vm-test-run-nixbot-gitlab> gitlab # [  305.427145] gitaly[1383]: time="2026-07-04T10:54:52.258Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-04T10:55:22.144" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:52.244" grpc.time_ms=13.782 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985vm-test-run-nixbot-gitlab> gitlab # [  305.456603] gitaly[1383]: time="2026-07-04T10:54:52.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:02.173" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:52.274" grpc.time_ms=13.772 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986vm-test-run-nixbot-gitlab> gitlab # [  305.566158] gitaly[1383]: time="2026-07-04T10:54:52.396Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-04T10:55:02.295" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:52.396" grpc.time_ms=0.593 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987vm-test-run-nixbot-gitlab> gitlab # [  305.601825] gitaly[1383]: time="2026-07-04T10:54:52.433Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-04T10:55:02.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:52.415" grpc.time_ms=17.372 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988vm-test-run-nixbot-gitlab> gitlab # [  305.679024] gitaly[1383]: time="2026-07-04T10:54:52.510Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:52.509" grpc.time_ms=0.866 pid=1383 span.kind=server system=grpc1989vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f'1990vm-test-run-nixbot-gitlab> gitlab # [  305.731319] gitaly[1383]: time="2026-07-04T10:54:52.562Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:52.561" grpc.time_ms=0.777 pid=1383 span.kind=server system=grpc1991vm-test-run-nixbot-gitlab> gitlab # [  305.758575] gitaly[1383]: time="2026-07-04T10:54:52.573Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC diskcache=130438f5-319a-4be6-bda0-d57cd97e8948 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T10:55:02.026" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:52.437" pid=1383 ref=refs/keep-around/7f17f820668d42138a4ec71e7024cc120878ae6f remote_ip="2001:db8:1::1" revision=7f17f820668d42138a4ec71e7024cc120878ae6f span.kind=server system=grpc username=root1992vm-test-run-nixbot-gitlab> gitlab # [  305.781324] gitaly[1383]: time="2026-07-04T10:54:52.574Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=138 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5193 command.oublock=8 command.real_time_ms=128 command.spawn_token_fork_ms=2 command.system_time_ms=82 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-04T10:55:02.026" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-04T10:54:52.437" grpc.time_ms=136.407 pid=1383 ref=refs/keep-around/7f17f820668d42138a4ec71e7024cc120878ae6f remote_ip="2001:db8:1::1" revision=7f17f820668d42138a4ec71e7024cc120878ae6f span.kind=server system=grpc username=root1993vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f', in 0.09 seconds)1994vm-test-run-nixbot-gitlab> [{'id': 1, 'project_id': 1, 'number': 1, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-04T10:54:52.150446Z', 'started_at': None, 'finished_at': None}]1995vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 3.69 seconds)1996vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses1997vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses1998vm-test-run-nixbot-gitlab> gitlab # [  305.803696] gitaly[1383]: time="2026-07-04T10:54:52.580Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:22.479" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:52.579" grpc.time_ms=0.7 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999vm-test-run-nixbot-gitlab> gitlab # [  305.822782] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.618Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"12e7d055575168ea61317eee","created_at":"2026-07-04T10:54:52.594Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.611Z","job_size_bytes":65,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-12e7d055575168ea61317eee: start","job_status":"start","queue_duration_s":0.006511,"scheduling_latency_s":0.006576}2000vm-test-run-nixbot-gitlab> gitlab # [  305.850266] nixbot[3780]: INFO:     ::ffff:127.0.0.1:47306 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 200 OK2001vm-test-run-nixbot-gitlab> gitlab # [  305.925512] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.755Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"12e7d055575168ea61317eee","created_at":"2026-07-04T10:54:52.594Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.611Z","job_size_bytes":65,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-12e7d055575168ea61317eee: done: 0.136976 sec","job_status":"done","queue_duration_s":0.006511,"scheduling_latency_s":0.006576,"redis_calls":19,"redis_duration_s":0.013391,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.009645,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002929,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000817,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058046,"duration_s":0.136976,"completed_at":"2026-07-04T10:54:52.754Z","load_balancing_strategy":"primary","db_duration_s":0.023762,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2002vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.20 seconds)2003vm-test-run-nixbot-gitlab> []2004vm-test-run-nixbot-gitlab> gitlab # [  306.012767] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.760Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c304443f46b89dbfef09cd36","created_at":"2026-07-04T10:54:52.617Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.631Z","job_size_bytes":65,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c304443f46b89dbfef09cd36: start","job_status":"start","queue_duration_s":0.129575,"scheduling_latency_s":0.129605}2005vm-test-run-nixbot-gitlab> gitlab # [  306.037277] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 1172006vm-test-run-nixbot-gitlab> gitlab # [  306.040964] gitaly[1383]: time="2026-07-04T10:54:52.774Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8NRZAMDTDXBVGZGGH3AG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:22.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:52.772" grpc.time_ms=2.101 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2007vm-test-run-nixbot-gitlab> gitlab # [  306.058238] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.852Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c304443f46b89dbfef09cd36","created_at":"2026-07-04T10:54:52.617Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.631Z","job_size_bytes":65,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c304443f46b89dbfef09cd36: done: 0.091602 sec","job_status":"done","queue_duration_s":0.129575,"scheduling_latency_s":0.129605,"redis_calls":21,"redis_duration_s":0.016829,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.007822,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007581,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002532,"duration_s":0.091602,"completed_at":"2026-07-04T10:54:52.852Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008vm-test-run-nixbot-gitlab> gitlab # [  306.111692] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:52.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0e3745ac473259d066cea649","created_at":"2026-07-04T10:54:52.848Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.854Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0e3745ac473259d066cea649: start","job_status":"start","queue_duration_s":0.000886,"scheduling_latency_s":0.000909}2009vm-test-run-nixbot-gitlab> gitlab # [  306.127910] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 7172010vm-test-run-nixbot-gitlab> gitlab # [  306.130587] nixbot[3780]: {"ts": "2026-07-04T10:54:52.873Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f \"HTTP/1.1 201 Created\""}2011vm-test-run-nixbot-gitlab> gitlab # [  306.133496] nixbot[3780]: {"ts": "2026-07-04T10:54:52.889Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-7f17f820668d-53262555", "argv0": "bwrap"}2012vm-test-run-nixbot-gitlab> gitlab # [  306.135954] gitaly[1383]: time="2026-07-04T10:54:52.962Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T10:55:22.160" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:52.961" grpc.time_ms=1.188 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2013vm-test-run-nixbot-gitlab> gitlab # [  306.272790] systemd[1]: Started Nix Daemon.2014vm-test-run-nixbot-gitlab> gitlab # [  306.381915] gitaly[1383]: time="2026-07-04T10:54:53.213Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:23.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:53.211" grpc.time_ms=1.393 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2015vm-test-run-nixbot-gitlab> gitlab # [  306.406872] nix-daemon[4096]: accepted connection from pid 4095, user nixbot2016vm-test-run-nixbot-gitlab> gitlab # [  306.415333] nix-daemon[4096]: accepted connection from pid 4094, user nixbot2017vm-test-run-nixbot-gitlab> gitlab # [  306.440966] gitaly[1383]: time="2026-07-04T10:54:53.272Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8N7YG01TJBJ11B5WWAKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:23.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:53.271" grpc.time_ms=1.132 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018vm-test-run-nixbot-gitlab> gitlab # [  306.469284] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:53.292Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0e3745ac473259d066cea649","created_at":"2026-07-04T10:54:52.848Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-64cc27ed1b944588","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.854Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0e3745ac473259d066cea649: done: 0.437473 sec","job_status":"done","queue_duration_s":0.000886,"scheduling_latency_s":0.000909,"gitaly_calls":3,"gitaly_duration_s":0.017192,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.013651,"redis_read_bytes":235,"redis_write_bytes":7157,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001155,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003986,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001538,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000358,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002678,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000668,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000957,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001156,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.221,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.195,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.190845,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3968626,"duration_s":0.437473,"completed_at":"2026-07-04T10:54:53.292Z","load_balancing_strategy":"primary","db_duration_s":0.228589,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2019vm-test-run-nixbot-gitlab> gitlab # [  306.547905] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:53.295Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7708d0c2c59bc708a16b4a02","created_at":"2026-07-04T10:54:52.940Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-9e6ffa61ce5048cf","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.945Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7708d0c2c59bc708a16b4a02: start","job_status":"start","queue_duration_s":0.349283,"scheduling_latency_s":0.349313}2020vm-test-run-nixbot-gitlab> gitlab # [  306.565100] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:53.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7708d0c2c59bc708a16b4a02","created_at":"2026-07-04T10:54:52.940Z","trace_propagation_headers":{"sentry-trace":"cc506a685a5e465ba0058e2c7ddf693b-9e6ffa61ce5048cf","baggage":"sentry-trace_id=cc506a685a5e465ba0058e2c7ddf693b,sentry-release=Unknown"},"correlation_id":"01KWPC8N7YG01TJBJ11B5WWAKC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:52.945Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7708d0c2c59bc708a16b4a02: done: 0.04906 sec","job_status":"done","queue_duration_s":0.349283,"scheduling_latency_s":0.349313,"redis_calls":10,"redis_duration_s":0.002901,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000542,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004278,"duration_s":0.04906,"completed_at":"2026-07-04T10:54:53.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2022vm-test-run-nixbot-gitlab> gitlab # [  307.176737] gitaly[1383]: time="2026-07-04T10:54:54.007Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8PYH0RBTE2EWRZZKA9TD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:24.004" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:54.005" grpc.time_ms=1.271 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2023vm-test-run-nixbot-gitlab> gitlab # [  307.329551] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 2542024vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.32 seconds)2025vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-04T10:54:52.838Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2026vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2027vm-test-run-nixbot-gitlab> gitlab # [  308.482866] gitaly[1383]: time="2026-07-04T10:54:55.313Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8R7JNHX43JVY0P80880F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:25.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:55.311" grpc.time_ms=1.163 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.20 seconds)2029vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-04T10:54:52.838Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2030vm-test-run-nixbot-gitlab> gitlab # [  308.535643] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1472031vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2032vm-test-run-nixbot-gitlab> gitlab # [  309.686400] gitaly[1383]: time="2026-07-04T10:54:56.516Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8SD18XXWK7WB7K2B2M75 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:26.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:56.515" grpc.time_ms=1.293 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.20 seconds)2034vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-04T10:54:52.838Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2035vm-test-run-nixbot-gitlab> gitlab # [  309.735513] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 1482036vm-test-run-nixbot-gitlab> gitlab # [  310.179810] redis[1278]: 10 changes in 300 seconds. Saving...2037vm-test-run-nixbot-gitlab> gitlab # [  310.183221] redis[1278]: Background saving started by pid 41272038vm-test-run-nixbot-gitlab> gitlab # [  310.191075] redis[4127]: BGSAVE done, 546 keys saved, 0 keys skipped, 129837 bytes written.2039vm-test-run-nixbot-gitlab> gitlab # [  310.195842] redis[4127]: DB saved on disk2040vm-test-run-nixbot-gitlab> gitlab # [  310.198244] redis[4127]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB2041vm-test-run-nixbot-gitlab> gitlab # [  310.280930] redis[1278]: Background saving terminated with success2042vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2043vm-test-run-nixbot-gitlab> gitlab # [  310.964333] gitaly[1383]: time="2026-07-04T10:54:57.794Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TJM414FK35JMTSYN1ZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:27.192" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:57.793" grpc.time_ms=1.339 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044vm-test-run-nixbot-gitlab> gitlab # [  311.057461] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 2682045vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.32 seconds)2046vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-04T10:54:52.838Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2047vm-test-run-nixbot-gitlab> gitlab # [  311.063973] gitaly[1383]: time="2026-07-04T10:54:57.895Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:27.793" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:57.893" grpc.time_ms=1.019 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048vm-test-run-nixbot-gitlab> gitlab # [  311.154181] gitaly[1383]: time="2026-07-04T10:54:57.985Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:07.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:57.967" grpc.time_ms=18.053 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049vm-test-run-nixbot-gitlab> gitlab # [  311.234006] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.063Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c466a1806db7f69968171ef5","created_at":"2026-07-04T10:54:58.053Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-4d2e2ddc9ffe48d2","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.060Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c466a1806db7f69968171ef5: start","job_status":"start","queue_duration_s":0.002894,"scheduling_latency_s":0.002964}2050vm-test-run-nixbot-gitlab> gitlab # [  311.282652] nixbot[3780]: {"ts": "2026-07-04T10:54:58.090Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f \"HTTP/1.1 201 Created\""}2051vm-test-run-nixbot-gitlab> gitlab # [  311.289650] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 3052052vm-test-run-nixbot-gitlab> gitlab # [  311.491463] gitaly[1383]: time="2026-07-04T10:54:58.322Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:58.321" grpc.time_ms=1.102 pid=1383 span.kind=server system=grpc2053vm-test-run-nixbot-gitlab> gitlab # [  311.518881] gitaly[1383]: time="2026-07-04T10:54:58.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8V1J5E20TGMYRW1B6TWZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:28.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:58.335" grpc.time_ms=1.172 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2054vm-test-run-nixbot-gitlab> gitlab # [  311.544862] gitaly[1383]: time="2026-07-04T10:54:58.376Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:58.376" grpc.time_ms=0.3 pid=1383 span.kind=server system=grpc2055vm-test-run-nixbot-gitlab> gitlab # [  311.573434] gitaly[1383]: time="2026-07-04T10:54:58.405Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-04T10:54:58.404" grpc.time_ms=0.475 pid=1383 span.kind=server system=grpc2056vm-test-run-nixbot-gitlab> gitlab # [  311.592782] gitaly[1383]: time="2026-07-04T10:54:58.415Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 diskcache=389cf082-2081-40f1-8bf6-ef577a939cdb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T10:55:28.064" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:58.265" pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057vm-test-run-nixbot-gitlab> gitlab # [  311.610410] gitaly[1383]: time="2026-07-04T10:54:58.415Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=148 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7311 command.oublock=0 command.real_time_ms=143 command.spawn_token_fork_ms=1 command.system_time_ms=68 command.user_time_ms=80 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-04T10:55:28.064" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:58.265" grpc.time_ms=149.75 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058vm-test-run-nixbot-gitlab> gitlab # [  311.694396] gitaly[1383]: time="2026-07-04T10:54:58.526Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T10:55:28.324" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:58.524" grpc.time_ms=1.178 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059vm-test-run-nixbot-gitlab> gitlab # [  311.737168] gitaly[1383]: time="2026-07-04T10:54:58.568Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:28.366" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:58.567" grpc.time_ms=1.201 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060vm-test-run-nixbot-gitlab> gitlab # [  311.753609] gitaly[1383]: time="2026-07-04T10:54:58.585Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8V1J5E20TGMYRW1B6TWZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:08.056" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:58.567" grpc.time_ms=17.808 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061vm-test-run-nixbot-gitlab> gitlab # [  311.814546] gitaly[1383]: time="2026-07-04T10:54:58.646Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8TQTGDK7FHERGEVFT658 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:28.444" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:58.644" grpc.time_ms=1.223 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062vm-test-run-nixbot-gitlab> gitlab # [  311.864315] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.693Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c466a1806db7f69968171ef5","created_at":"2026-07-04T10:54:58.053Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-4d2e2ddc9ffe48d2","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.060Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c466a1806db7f69968171ef5: done: 0.630693 sec","job_status":"done","queue_duration_s":0.002894,"scheduling_latency_s":0.002964,"gitaly_calls":4,"gitaly_duration_s":0.198458,"redis_calls":110,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0334,"redis_read_bytes":195,"redis_write_bytes":25577,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001916,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003411,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":26,"redis_feature_flag_duration_s":0.007886,"redis_feature_flag_read_bytes":26,"redis_feature_flag_write_bytes":4583,"redis_queues_calls":23,"redis_queues_duration_s":0.004545,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13381,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.009876,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000504,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000497,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002849,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":851,"db_count":49,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":49,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.082,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.035,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.039,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.037,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.337926,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.57374396,"duration_s":0.630693,"completed_at":"2026-07-04T10:54:58.693Z","load_balancing_strategy":"primary","db_duration_s":0.087659,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2063vm-test-run-nixbot-gitlab> gitlab # [  311.958672] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.697Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1763097af331ab1396e0035f","created_at":"2026-07-04T10:54:58.234Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.246Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1763097af331ab1396e0035f: start","job_status":"start","queue_duration_s":0.451253,"scheduling_latency_s":0.451281}2064vm-test-run-nixbot-gitlab> gitlab # [  311.978330] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"1763097af331ab1396e0035f","created_at":"2026-07-04T10:54:58.234Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.246Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-1763097af331ab1396e0035f: done: 0.094658 sec","job_status":"done","queue_duration_s":0.451253,"scheduling_latency_s":0.451281,"redis_calls":10,"redis_duration_s":0.005774,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0014074,"duration_s":0.094658,"completed_at":"2026-07-04T10:54:58.791Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020588,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2065vm-test-run-nixbot-gitlab> gitlab # [  312.028672] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.795Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"33329141cffe45bf2044328a","created_at":"2026-07-04T10:54:58.422Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.431Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-33329141cffe45bf2044328a: start","job_status":"start","queue_duration_s":0.363538,"scheduling_latency_s":0.363558}2066vm-test-run-nixbot-gitlab> gitlab # [  312.047164] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 6992067vm-test-run-nixbot-gitlab> gitlab # [  312.049891] nixbot[3780]: {"ts": "2026-07-04T10:54:58.800Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f \"HTTP/1.1 201 Created\""}2068vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2069vm-test-run-nixbot-gitlab> gitlab # [  312.052777] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.832Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"33329141cffe45bf2044328a","created_at":"2026-07-04T10:54:58.422Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.431Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-33329141cffe45bf2044328a: done: 0.03795 sec","job_status":"done","queue_duration_s":0.363538,"scheduling_latency_s":0.363558,"redis_calls":10,"redis_duration_s":0.002541,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036442,"duration_s":0.03795,"completed_at":"2026-07-04T10:54:58.832Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070vm-test-run-nixbot-gitlab> gitlab # [  312.097184] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"774a2e150b1a16f306ec423b","created_at":"2026-07-04T10:54:58.452Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.462Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-774a2e150b1a16f306ec423b: start","job_status":"start","queue_duration_s":0.37165,"scheduling_latency_s":0.371667}2071vm-test-run-nixbot-gitlab> gitlab # [  312.113263] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.879Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"774a2e150b1a16f306ec423b","created_at":"2026-07-04T10:54:58.452Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.462Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-774a2e150b1a16f306ec423b: done: 0.045097 sec","job_status":"done","queue_duration_s":0.37165,"scheduling_latency_s":0.371667,"redis_calls":10,"redis_duration_s":0.002575,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000497,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001539,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036472,"duration_s":0.045097,"completed_at":"2026-07-04T10:54:58.879Z","load_balancing_strategy":"primary","db_duration_s":0.013997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072vm-test-run-nixbot-gitlab> gitlab # [  312.159745] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ae41e7a889e5db1ac2a5aa98","created_at":"2026-07-04T10:54:58.480Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.486Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ae41e7a889e5db1ac2a5aa98: start","job_status":"start","queue_duration_s":0.394163,"scheduling_latency_s":0.394181}2073vm-test-run-nixbot-gitlab> gitlab # [  312.175598] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.922Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"ae41e7a889e5db1ac2a5aa98","created_at":"2026-07-04T10:54:58.480Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.486Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-ae41e7a889e5db1ac2a5aa98: done: 0.040985 sec","job_status":"done","queue_duration_s":0.394163,"scheduling_latency_s":0.394181,"redis_calls":13,"redis_duration_s":0.0041789999999999996,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002303,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000847,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000534,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00084404,"duration_s":0.040985,"completed_at":"2026-07-04T10:54:58.922Z","load_balancing_strategy":"primary","db_duration_s":0.009049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2074vm-test-run-nixbot-gitlab> gitlab # [  312.221751] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bd21a4969649ca04425e5520","created_at":"2026-07-04T10:54:58.487Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.495Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bd21a4969649ca04425e5520: start","job_status":"start","queue_duration_s":0.431797,"scheduling_latency_s":0.431819}2075vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.20 seconds)2076vm-test-run-nixbot-gitlab> [{'id': 1, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-04T10:54:52.838Z', 'started_at': None, 'finished_at': '2026-07-04T10:54:58.033Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'building attributes', 'created_at': '2026-07-04T10:54:58.671Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2077vm-test-run-nixbot-gitlab> gitlab # [  312.239179] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.973Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bd21a4969649ca04425e5520","created_at":"2026-07-04T10:54:58.487Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.495Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bd21a4969649ca04425e5520: done: 0.044817 sec","job_status":"done","queue_duration_s":0.431797,"scheduling_latency_s":0.431819,"redis_calls":10,"redis_duration_s":0.002131,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003498,"duration_s":0.044817,"completed_at":"2026-07-04T10:54:58.971Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011055,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078vm-test-run-nixbot-gitlab> gitlab # [  312.285733] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:58.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5732fa26447d2bd86fd345e0","created_at":"2026-07-04T10:54:58.495Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.502Z","job_size_bytes":19,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5732fa26447d2bd86fd345e0: start","job_status":"start","queue_duration_s":0.472381,"scheduling_latency_s":0.4724}2079vm-test-run-nixbot-gitlab> gitlab # [  312.302745] postgres[1345]: [1345] LOG:  checkpoint starting: time2080vm-test-run-nixbot-gitlab> gitlab # [  312.303935] gitaly[1383]: time="2026-07-04T10:54:59.000Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8VR56SCEXH5B6D3YVJR3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:28.399" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:58.999" grpc.time_ms=0.539 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2081vm-test-run-nixbot-gitlab> gitlab # [  312.317231] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.022Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5732fa26447d2bd86fd345e0","created_at":"2026-07-04T10:54:58.495Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.502Z","job_size_bytes":19,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5732fa26447d2bd86fd345e0: done: 0.048201 sec","job_status":"done","queue_duration_s":0.472381,"scheduling_latency_s":0.4724,"redis_calls":11,"redis_duration_s":0.011325,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003141,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006997,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027538,"duration_s":0.048201,"completed_at":"2026-07-04T10:54:59.022Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082vm-test-run-nixbot-gitlab> gitlab # [  312.362520] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.023Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"38dd49fef73f01aeaacb0d14","created_at":"2026-07-04T10:54:58.505Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.512Z","job_size_bytes":66,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-38dd49fef73f01aeaacb0d14: start","job_status":"start","queue_duration_s":0.511098,"scheduling_latency_s":0.511115}2083vm-test-run-nixbot-gitlab> gitlab # [  312.380116] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.074Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"38dd49fef73f01aeaacb0d14","created_at":"2026-07-04T10:54:58.505Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.512Z","job_size_bytes":66,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-38dd49fef73f01aeaacb0d14: done: 0.050264 sec","job_status":"done","queue_duration_s":0.511098,"scheduling_latency_s":0.511115,"redis_calls":10,"redis_duration_s":0.010195000000000001,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000702,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001522,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00100222,"duration_s":0.050264,"completed_at":"2026-07-04T10:54:59.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084vm-test-run-nixbot-gitlab> gitlab # [  312.426118] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"62620430047ee39aca661ebe","created_at":"2026-07-04T10:54:58.514Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.516Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-62620430047ee39aca661ebe: start","job_status":"start","queue_duration_s":0.563875,"scheduling_latency_s":0.563919}2085vm-test-run-nixbot-gitlab> gitlab # [  312.442901] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"62620430047ee39aca661ebe","created_at":"2026-07-04T10:54:58.514Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.516Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-62620430047ee39aca661ebe: done: 0.035746 sec","job_status":"done","queue_duration_s":0.563875,"scheduling_latency_s":0.563919,"redis_calls":7,"redis_duration_s":0.0011099999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035746,"completed_at":"2026-07-04T10:54:59.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086vm-test-run-nixbot-gitlab> gitlab # [  312.487978] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.117Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0c081e272c163bb3c89951f6","created_at":"2026-07-04T10:54:58.576Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.585Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0c081e272c163bb3c89951f6: start","job_status":"start","queue_duration_s":0.53118,"scheduling_latency_s":0.531198}2087vm-test-run-nixbot-gitlab> gitlab # [  312.504889] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 1072088vm-test-run-nixbot-gitlab> gitlab # [  312.507564] gitaly[1383]: time="2026-07-04T10:54:59.040Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWPC8VWWCB6WG5S2X32ANNV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-04T10:55:29.039" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:59.040" grpc.time_ms=0.496 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2089vm-test-run-nixbot-gitlab> gitlab # [  312.520826] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.160Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0c081e272c163bb3c89951f6","created_at":"2026-07-04T10:54:58.576Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.585Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0c081e272c163bb3c89951f6: done: 0.043556 sec","job_status":"done","queue_duration_s":0.53118,"scheduling_latency_s":0.531198,"redis_calls":10,"redis_duration_s":0.001487,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002602,"duration_s":0.043556,"completed_at":"2026-07-04T10:54:59.160Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090vm-test-run-nixbot-gitlab> gitlab # [  312.565847] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.162Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dd45113314ab47835cc245b1","created_at":"2026-07-04T10:54:58.587Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.592Z","job_size_bytes":6,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dd45113314ab47835cc245b1: start","job_status":"start","queue_duration_s":0.569639,"scheduling_latency_s":0.56966}2091vm-test-run-nixbot-gitlab> gitlab # [  312.582291] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dd45113314ab47835cc245b1","created_at":"2026-07-04T10:54:58.587Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.592Z","job_size_bytes":6,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dd45113314ab47835cc245b1: done: 0.029933 sec","job_status":"done","queue_duration_s":0.569639,"scheduling_latency_s":0.56966,"redis_calls":10,"redis_duration_s":0.002582,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000692,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004893,"duration_s":0.029933,"completed_at":"2026-07-04T10:54:59.191Z","load_balancing_strategy":"primary","db_duration_s":0.004071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092vm-test-run-nixbot-gitlab> gitlab # [  312.626393] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e0d04d5587ddb3e28f530d2a","created_at":"2026-07-04T10:54:58.593Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.597Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e0d04d5587ddb3e28f530d2a: start","job_status":"start","queue_duration_s":0.595993,"scheduling_latency_s":0.596013}2093vm-test-run-nixbot-gitlab> gitlab # [  312.642228] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e0d04d5587ddb3e28f530d2a","created_at":"2026-07-04T10:54:58.593Z","trace_propagation_headers":{"sentry-trace":"d697be71ede44140a805af87118ea791-a9e07fff522148b3","baggage":"sentry-trace_id=d697be71ede44140a805af87118ea791,sentry-release=Unknown"},"correlation_id":"01KWPC8TQTGDK7FHERGEVFT658","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.597Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e0d04d5587ddb3e28f530d2a: done: 0.033032 sec","job_status":"done","queue_duration_s":0.595993,"scheduling_latency_s":0.596013,"redis_calls":10,"redis_duration_s":0.00247,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002715,"duration_s":0.033032,"completed_at":"2026-07-04T10:54:59.226Z","load_balancing_strategy":"primary","db_duration_s":0.005162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2094vm-test-run-nixbot-gitlab> gitlab # [  312.687680] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.228Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f5cf333d114dc4585db45af","created_at":"2026-07-04T10:54:58.706Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-f3b047e335064896","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.731Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f5cf333d114dc4585db45af: start","job_status":"start","queue_duration_s":0.49727,"scheduling_latency_s":0.49729}2095vm-test-run-nixbot-gitlab> gitlab # [  312.704100] gitaly[1383]: time="2026-07-04T10:54:59.174Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8VR56SCEXH5B6D3YVJR3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-04T10:55:09.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-04T10:54:59.158" grpc.time_ms=16.073 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2096vm-test-run-nixbot-gitlab> gitlab # [  312.717509] gitlab-workhorse[1302]: gitlab 2001:db8:1::1 - - [2026/07/04:10:54:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 5302097vm-test-run-nixbot-gitlab> gitlab # [  312.720402] nixbot[3780]: {"ts": "2026-07-04T10:54:59.349Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7f17f820668d42138a4ec71e7024cc120878ae6f \"HTTP/1.1 201 Created\""}2098vm-test-run-nixbot-gitlab> gitlab # [  312.723218] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.428Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2010029b349436b5da16735d","created_at":"2026-07-04T10:54:59.422Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-da070e0afe5848f9","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"3f5cf333d114dc4585db45af","job_size_bytes":3,"pid":2431,"job_status":"deduplicated","message":"PipelineProcessWorker JID-2010029b349436b5da16735d: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2099vm-test-run-nixbot-gitlab> gitlab # [  312.739087] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.431Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f5cf333d114dc4585db45af","created_at":"2026-07-04T10:54:58.706Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-f3b047e335064896","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.731Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2431,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3f5cf333d114dc4585db45af: rescheduled"}2100vm-test-run-nixbot-gitlab> gitlab # [  312.754993] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.442Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3f5cf333d114dc4585db45af","created_at":"2026-07-04T10:54:58.706Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-f3b047e335064896","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:58.731Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3f5cf333d114dc4585db45af: done: 0.213453 sec","job_status":"done","queue_duration_s":0.49727,"scheduling_latency_s":0.49729,"gitaly_calls":3,"gitaly_duration_s":0.011866,"redis_calls":50,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01341,"redis_read_bytes":182,"redis_write_bytes":7762,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001245,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002481,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":4,"redis_queues_duration_s":0.000752,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2221,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.005485,"redis_queues_metadata_read_bytes":89,"redis_queues_metadata_write_bytes":1432,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00027,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000978,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.148018,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1691517,"duration_s":0.213453,"completed_at":"2026-07-04T10:54:59.442Z","load_balancing_strategy":"primary","db_duration_s":0.051527,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101vm-test-run-nixbot-gitlab> gitlab # [  312.807867] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.444Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"10e2d50a71659ec712d4680b","created_at":"2026-07-04T10:54:41.253Z","trace_propagation_headers":{"sentry-trace":"f798e46bbf9c4032b3943f0f77db1564-6454fd30caae4d76","baggage":"sentry-trace_id=f798e46bbf9c4032b3943f0f77db1564,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC89Z8Y0J18MMDTDK5CF6V","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2011343ad259f9b0f36611a4ad432a35e375d93940e8539715b411c1fade5d04","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.311Z","retry_count":0,"job_size_bytes":451,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-10e2d50a71659ec712d4680b: start","job_status":"start","queue_duration_s":0.133495,"scheduling_latency_s":0.133522}2102vm-test-run-nixbot-gitlab> gitlab # [  312.825142] sidekiq-cluster[2431]: {"severity":"WARN","time":"2026-07-04T10:54:59.491Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"10e2d50a71659ec712d4680b","created_at":"2026-07-04T10:54:41.253Z","trace_propagation_headers":{"sentry-trace":"f798e46bbf9c4032b3943f0f77db1564-6454fd30caae4d76","baggage":"sentry-trace_id=f798e46bbf9c4032b3943f0f77db1564,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWPC89Z8Y0J18MMDTDK5CF6V","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:2011343ad259f9b0f36611a4ad432a35e375d93940e8539715b411c1fade5d04","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.311Z","retry_count":0,"job_size_bytes":451,"pid":2431,"sidekiq_tid":"m4ff","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-10e2d50a71659ec712d4680b: fail: 0.046548 sec","job_status":"fail","queue_duration_s":0.133495,"scheduling_latency_s":0.133522,"redis_calls":7,"redis_duration_s":0.001459,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001051,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046548,"completed_at":"2026-07-04T10:54:59.491Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/anil03slywxmk4rnhn17xdngk0v51799-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/rwla413y2rjqwmj6fahb1jjs3246g6h0-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/w9mp1gyfs2qdv12r0ji70idzndmxwz2s-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/r9cfijmkxqn74a9y8n4fmcwypmha7bka-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/p8ylyr0sj9sxcknqcpja13nypqixk2dp-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/dmv1shq667w4j882d6hfknpqjgmsjk3d-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/dndr7b4a17339dsm67vzb6nv4p2kp82q-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/8bxd3x89628dxvamnlkr67i03nyww454-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/hci32c03dmcimq95f7j6mhr8r3jxmcw2-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/qr8jwlszzf80b9yy5410msfvxfhdlv3q-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/zx6ndpscxs64hnidz69ab430ccms6rak-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/cf9hps81b6xv5b1r3vmw17kgnjg02fib-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/q50z9k7w21nl4781zv782gv6pin0gj9b-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.006917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103vm-test-run-nixbot-gitlab> gitlab # [  313.228477] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.496Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"88a515bfb978c14b510758c5","created_at":"2026-07-04T10:54:59.296Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-da070e0afe5848f9","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.298Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-88a515bfb978c14b510758c5: start","job_status":"start","queue_duration_s":0.197285,"scheduling_latency_s":0.197335}2104vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses2105vm-test-run-nixbot-gitlab> gitlab # [  313.245158] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.542Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"88a515bfb978c14b510758c5","created_at":"2026-07-04T10:54:59.296Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-da070e0afe5848f9","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.298Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-88a515bfb978c14b510758c5: done: 0.046217 sec","job_status":"done","queue_duration_s":0.197285,"scheduling_latency_s":0.197335,"redis_calls":7,"redis_duration_s":0.002594,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046217,"completed_at":"2026-07-04T10:54:59.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106vm-test-run-nixbot-gitlab> gitlab # [  313.287216] gitaly[1383]: time="2026-07-04T10:54:59.325Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWPC8V1J5E20TGMYRW1B6TWZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-04T10:55:29.123" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-04T10:54:59.324" grpc.time_ms=1.166 pid=1383 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2107vm-test-run-nixbot-gitlab> gitlab # [  313.300913] nixbot[3780]: {"ts": "2026-07-04T10:55:00.093Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "cefc112e8680bb0f9569544b9656644ac3312d93"}2108vm-test-run-nixbot-gitlab> gitlab # [  313.303457] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.544Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"3454be06cf3c202fa10ec18d","created_at":"2026-07-04T10:54:59.431Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-da070e0afe5848f9","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.434Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3454be06cf3c202fa10ec18d: start","job_status":"start","queue_duration_s":0.109408,"scheduling_latency_s":0.109427}2109vm-test-run-nixbot-gitlab> gitlab # [  313.320554] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.900Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"3454be06cf3c202fa10ec18d","created_at":"2026-07-04T10:54:59.431Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-da070e0afe5848f9","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.434Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3454be06cf3c202fa10ec18d: done: 0.356136 sec","job_status":"done","queue_duration_s":0.109408,"scheduling_latency_s":0.109427,"gitaly_calls":4,"gitaly_duration_s":0.11084,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.03099,"redis_read_bytes":173,"redis_write_bytes":20928,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001234,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002843,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.012769,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13381,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008786,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00029,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00126,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002574,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187162,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.322888,"duration_s":0.356136,"completed_at":"2026-07-04T10:54:59.900Z","load_balancing_strategy":"primary","db_duration_s":0.039144,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2110vm-test-run-nixbot-gitlab> gitlab # [  313.373044] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.901Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"38ff08b28bc472baa4c5c7fd","created_at":"2026-07-04T10:54:59.610Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.613Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-38ff08b28bc472baa4c5c7fd: start","job_status":"start","queue_duration_s":0.288085,"scheduling_latency_s":0.288112}2111vm-test-run-nixbot-gitlab> gitlab # [  313.390222] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"38ff08b28bc472baa4c5c7fd","created_at":"2026-07-04T10:54:59.610Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.613Z","job_size_bytes":23,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-38ff08b28bc472baa4c5c7fd: done: 0.047833 sec","job_status":"done","queue_duration_s":0.288085,"scheduling_latency_s":0.288112,"redis_calls":7,"redis_duration_s":0.001045,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047833,"completed_at":"2026-07-04T10:54:59.949Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011402,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2112vm-test-run-nixbot-gitlab> gitlab # [  313.434313] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.950Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5c1cccd0622f1744ea399b10","created_at":"2026-07-04T10:54:59.715Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.720Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5c1cccd0622f1744ea399b10: start","job_status":"start","queue_duration_s":0.230651,"scheduling_latency_s":0.230678}2113vm-test-run-nixbot-gitlab> gitlab # [  313.451611] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.977Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5c1cccd0622f1744ea399b10","created_at":"2026-07-04T10:54:59.715Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.720Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5c1cccd0622f1744ea399b10: done: 0.026365 sec","job_status":"done","queue_duration_s":0.230651,"scheduling_latency_s":0.230678,"redis_calls":7,"redis_duration_s":0.001098,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026365,"completed_at":"2026-07-04T10:54:59.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114vm-test-run-nixbot-gitlab> gitlab # [  313.495635] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:54:59.978Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"17dab615526d4f6300045beb","created_at":"2026-07-04T10:54:59.729Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.740Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-17dab615526d4f6300045beb: start","job_status":"start","queue_duration_s":0.23792,"scheduling_latency_s":0.237947}2115vm-test-run-nixbot-gitlab> gitlab # [  313.512129] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.012Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"17dab615526d4f6300045beb","created_at":"2026-07-04T10:54:59.729Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.740Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-17dab615526d4f6300045beb: done: 0.03362 sec","job_status":"done","queue_duration_s":0.23792,"scheduling_latency_s":0.237947,"redis_calls":7,"redis_duration_s":0.00109,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000304,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03362,"completed_at":"2026-07-04T10:55:00.012Z","load_balancing_strategy":"primary","db_duration_s":0.007605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2116vm-test-run-nixbot-gitlab> gitlab # [  313.556590] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.013Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"beb6e2a04543c5d6858aaf5d","created_at":"2026-07-04T10:54:59.757Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.760Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-beb6e2a04543c5d6858aaf5d: start","job_status":"start","queue_duration_s":0.253459,"scheduling_latency_s":0.253484}2117vm-test-run-nixbot-gitlab> gitlab # [  313.572146] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"beb6e2a04543c5d6858aaf5d","created_at":"2026-07-04T10:54:59.757Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.760Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-beb6e2a04543c5d6858aaf5d: done: 0.029432 sec","job_status":"done","queue_duration_s":0.253459,"scheduling_latency_s":0.253484,"redis_calls":10,"redis_duration_s":0.0027949999999999997,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000886,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000728,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029432,"completed_at":"2026-07-04T10:55:00.043Z","load_balancing_strategy":"primary","db_duration_s":0.003195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2118vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7f17f820668d42138a4ec71e7024cc120878ae6f/statuses, in 0.37 seconds)2119vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': '0 attributes built', 'created_at': '2026-07-04T10:54:58.671Z', 'started_at': None, 'finished_at': '2026-07-04T10:54:59.260Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 3, 'sha': '7f17f820668d42138a4ec71e7024cc120878ae6f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded', 'created_at': '2026-07-04T10:55:00.337Z', 'started_at': None, 'finished_at': '2026-07-04T10:55:00.334Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2120vm-test-run-nixbot-gitlab> gitlab # [  313.616722] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1e751526000874a2c0546e2f","created_at":"2026-07-04T10:54:59.760Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.762Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1e751526000874a2c0546e2f: start","job_status":"start","queue_duration_s":0.282056,"scheduling_latency_s":0.282081}2121vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 7.82 seconds)2122vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 379.12 seconds)2123vm-test-run-nixbot-gitlab> gitlab # [  313.632546] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1e751526000874a2c0546e2f","created_at":"2026-07-04T10:54:59.760Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.762Z","job_size_bytes":3,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1e751526000874a2c0546e2f: done: 0.031989 sec","job_status":"done","queue_duration_s":0.282056,"scheduling_latency_s":0.282081,"redis_calls":7,"redis_duration_s":0.0031980000000000003,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031989,"completed_at":"2026-07-04T10:55:00.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124vm-test-run-nixbot-gitlab> gitlab # [  313.679443] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.080Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"38462958037321dfe90051c5","created_at":"2026-07-04T10:54:59.763Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.766Z","job_size_bytes":19,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-38462958037321dfe90051c5: start","job_status":"start","queue_duration_s":0.314534,"scheduling_latency_s":0.314568}2125vm-test-run-nixbot-gitlab> gitlab # [  313.697939] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"38462958037321dfe90051c5","created_at":"2026-07-04T10:54:59.763Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.766Z","job_size_bytes":19,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-38462958037321dfe90051c5: done: 0.027372 sec","job_status":"done","queue_duration_s":0.314534,"scheduling_latency_s":0.314568,"redis_calls":8,"redis_duration_s":0.0013340000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027372,"completed_at":"2026-07-04T10:55:00.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126vm-test-run-nixbot-gitlab> test script finished in 379.25s2127vm-test-run-nixbot-gitlab> cleanup2128vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13)2129vm-test-run-nixbot-gitlab> gitlab # [  313.745647] sidekiq-cluster[2431]: {"severity":"INFO","time":"2026-07-04T10:55:00.109Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3f7ed050f940ab0777d503e4","created_at":"2026-07-04T10:54:59.767Z","trace_propagation_headers":{"sentry-trace":"028ce9203823411fbec8b537fe006d81-ffbac1d5ea424a8c","baggage":"sentry-trace_id=028ce9203823411fbec8b537fe006d81,sentry-release=Unknown"},"correlation_id":"01KWPC8V1J5E20TGMYRW1B6TWZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-04T10:54:59.769Z","job_size_bytes":66,"pid":2431,"sidekiq_tid":"m96z","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3f7ed050f940ab0777d503e4: start","job_status":"start","queue_duration_s":0.339458,"scheduling_latency_s":0.339475}2130vm-test-run-nixbot-gitlab> gitlab # [  313.766194] nix-daemon[4096]: accepted connection from pid 4181, user nixbot2131vm-test-run-nixbot-gitlab> gitlab # qemu-system-aarch64: terminating on signal 15 from pid 6 (/nix/store/lqn6mbgzzdrqq2qkwddcmxj9z6amdd86-python3-3.13.13/bin/python3.13)2132vm-test-run-nixbot-gitlab> (finished: cleanup, in 1.56 seconds)21332134post-build step Upload coverage to codecov: ok2135Skipping codecov: project=Mic92/nixbot attr=aarch64-linux.nixbot-gitlab